diff --git a/README.md b/README.md index 97eddc23..79b4ccfd 100644 --- a/README.md +++ b/README.md @@ -1,199 +1,213 @@ -# aiXplain SDK +# Welcome to aiXplain + +**The Agentic Operating System for Enterprise AI** + +aiXplain is a full-stack platform for building, deploying, and governing mission-critical AI agents at scale. With the aiXplain SDK, you can ship production-grade agents faster: + +- **Discover & connect** — Access hundreds of LLMs, tools, and integrations with a unified API, or bring your own. +- **Build & orchestrate** — Start from simple automations to **adaptive multi-agent systems** that reason, plan, and use tools, with a built-in memory. +- **Ground & retrieve** — Enhance agents with vector- and graph-based retrieval for accurate, context-aware responses. +- **Deploy anywhere** — Deploy with a click and let aiXplain handle the infrastructure (SaaS, on-prem, VPC) and MLOps so your agents can scale and evolve seamlessly. +- **Observe & improve** — Track usage and performance with tracing and audit trails, with enterprise-grade governance and compliance. + +aiXplain combines developer agility with enterprise-grade reliability in a platform where data sovereignty and compliance are non-negotiable. + +> Check out this benchmark: aiXplain's orchestration engine [outperforms](https://aixplain.com/blog/math-solving-agent-aixplain-vs-crewai-vs-autogen/) other agentic frameworks on complex tasks while balancing speed and cost. + +--- + +## aiXplain agents + +aiXplain agents are designed with built-in intelligence, a.k.a **microagents**, that handle the operational complexity of agents at runtime — such as planning, monitoring, validation, routing, and formatting. This frees you to focus on tuning your agents for your use case instead of rebuilding the basics.
- aiXplain logo -
-
- - [![Python 3.5+](https://img.shields.io/badge/python-3.5+-blue.svg)](https://www.python.org/downloads/) - [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](http://www.apache.org/licenses/LICENSE-2.0) - [![PyPI version](https://badge.fury.io/py/aiXplain.svg)](https://badge.fury.io/py/aiXplain) - - **The professional AI SDK for developers and enterprises** + aiXplain Workflow
-## 📖 API Reference +The diagram illustrates how the orchestration engine coordinates agents at runtime, enabling agents that are modular, traceable, and production-ready. -- **Complete documentation:** - - [Python](https://docs.aixplain.com/api-reference/python/) - - [Swift](https://docs.aixplain.com/api-reference/swift/) +### Microagents -## 🚀 Overview +Microagents are specialized components that manage core operational functions: -The aiXplain SDK is a comprehensive Python library that empowers developers to integrate cutting-edge AI capabilities into their applications with ease. Access thousands of AI models, build custom pipelines, and deploy intelligent solutions at scale. +- **Mentalist** — planning and goal decomposition +- **Orchestrator** — task routing and role assignment +- **Inspector** — validation and policy enforcement (e.g., PII redaction) +- **Bodyguard** — data access, privacy, and security enforcement +- **Responder** — formatting and output delivery -### ✨ Key Features +Microagents are highly configurable — from lightweight automations to complex, iterative systems — and appear in agent traces for easier debugging, auditing, and explainability. -- **🔍 Discover**: Access 35,000+ ready-to-use AI models across multiple domains -- **⚡ Benchmark**: Compare AI systems using comprehensive datasets and metrics -- **🛠️ Design**: Create and deploy custom AI pipelines with our visual designer -- **🎯 FineTune**: Enhance pre-trained models with your data for optimal performance +### Meta-agents -## 📦 Installation +Meta-agents boost adaptability by improving agent performance. The **Evolver** (in private beta) attaches to any agent, monitors KPIs and feedback, and refines behavior — also serving as a powerful benchmarking tool by simulating users and environments. -### Basic Installation -```bash -pip install aixplain -``` +### Orchestration modes -### With Model Building Support -```bash -pip install aixplain[model-builder] -``` +aiXplain agents support two orchestration modes: + +- **Static** — define tasks (`AgentTasks`) and order for deterministic, repeatable execution. +- **Dynamic** (default) — the **Mentalist** generates the execution plan at runtime for adaptive, context-aware responses. -## 🔑 Authentication +aiXplain also supports [pipelines](https://docs.aixplain.com/concepts/assets/pipelines/) — sequential workflows that connect models and tools in a fixed order. -Get your API key from the aiXplain platform: +--- -1. Visit [platform.aixplain.com](https://platform.aixplain.com) -2. Navigate to your API Keys section -3. Generate a new Team API key +## How to start? -### Set Your API Key +- **For technical teams** → Install the SDK and start building: -**Linux/macOS:** ```bash -export TEAM_API_KEY=your_api_key_here +pip install aixplain ``` -**Windows:** -```cmd -set TEAM_API_KEY=your_api_key_here +- **For business teams without technical resources** → [Contact aiXplain](https://aixplain.com/adaptable-ai/). Our **aiXperts** will help you develop your agentic solutions and deploy them on your choice of infrastructure. + +--- + +## Quick start + +### Installation + ``` +pip install aixplain +``` + +### Authentication -**Jupyter Notebook:** ```python -%env TEAM_API_KEY=your_api_key_here +import os +os.environ["AIXPLAIN_API_KEY"] = "" ``` -For detailed setup instructions, visit [docs.aixplain.com/setup](https://docs.aixplain.com/setup). +Get your API key from your [aiXplain account](https://console.aixplain.com/settings/keys). -## 🏃‍♂️ Quick Start +### Create and Run Your First Agent -### Running Your First Model +**Example:** A weather agent powered by the [Open Weather API](https://platform.aixplain.com/discover/model/66f83c216eb563266175e201) from the aiXplain marketplace. -```python -from aixplain.factories import ModelFactory +By default, aiXplain agents run on [GPT-4o-mini](https://platform.aixplain.com/discover/model/669a63646eb56306647e1091) as the reasoning model. You can swap it with any other model from the aiXplain marketplace at any time. -# Get an English to French translation model -model = ModelFactory.get("61dc52976eb5634cf06e97cc") +```python +from aixplain.factories import AgentFactory, ModelFactory -# Run translation -result = model.run("Hello, how are you today?") -print(result.data) # "Bonjour, comment allez-vous aujourd'hui?" -``` +# Add tools +weather_tool = ModelFactory.get("66f83c216eb563266175e201") # Tool ID for Open Weather API tools -### Building a Pipeline +# Create the agent +agent = AgentFactory.create( +name="Weather Agent", +description="An agent that answers queries about the current weather.", +instructions="Use the provided tool to answer weather queries.", +tools=[weather_tool], +) -```python -from aixplain.factories.pipeline_factory import PipelineFactory -from aixplain.modules.pipeline.designer import Input +# Run and test your agent +query = "What is the weather in Liverpool, UK?" +agent_response = agent.run(query) -pipeline = PipelineFactory.init("Multi Input-Output Pipeline") -text_input_node = Input(data="text_input", data_types=["TEXT"], pipeline=pipeline) +print(agent_response['data']['output']) +``` -TRANSLATION_ASSET_ID = '60ddefbe8d38c51c5885f98a' -translation_node = pipeline.translation(asset_id=TRANSLATION_ASSET_ID) +Find a wide selection of LLMs and tools to power your agents by browsing our [marketplace](https://platform.aixplain.com/discover). -SENTIMENT_ASSET_ID = '61728750720b09325cbcdc36' -sentiment_node = pipeline.sentiment_analysis(asset_id=SENTIMENT_ASSET_ID) +### Access your deployed agent and API integration code -text_input_node.link(translation_node, 'input', 'text') -translation_node.link(sentiment_node, 'data', 'text') +Once your agent is deployed, you can view its API integration details and generated code by visiting: -translated_output_node = translation_node.use_output('data') -sentiment_output_node = sentiment_node.use_output('data') +[https://platform.aixplain.com/discover/agent/](https://platform.aixplain.com/discover/agent/) -pipeline.save() -outputs = pipeline.run({ - 'text_input': 'This is example text to translate.' -}) +Just replace `` in the URL with your actual agent identifier (agent.id). -print(outputs) -``` +### Build and deploy a Team Agent -### Working with Agents +A team agent orchestrates multiple specialized agents to solve complex problems. ```python -from aixplain.factories import AgentFactory +from aixplain.factories import TeamAgentFactory, AgentFactory +from aixplain.modules.agent.agent_task import AgentTask -agent = AgentFactory.create( - name="Google Search Agent", - description="A search agent", - instructions="Use Google Search to answer queries.", - tools=[ - AgentFactory.create_model_tool("65c51c556eb563350f6e1bb1") - ], - llm_id="669a63646eb56306647e1091" -) -response = agent.run("How can I help you today?") -``` +# Define tasks for specialized agents +scrape_task = AgentTask(name="scrape_website", description="Scrapes websites to extract information", expected_output="Scraped website output.") -## 📊 Core Modules +wiki_task = AgentTask(name="wiki_query", description="Queries wikipedia to answer user questions", expected_output="Queried results from wikipedia.") -| Module | Description | Documentation | -|--------|-------------|---------------| -| **Models** | Access 35,000+ AI models | [docs.aixplain.com/concepts/assets/models](https://docs.aixplain.com/concepts/assets/models/) | -| **Pipelines** | Build custom AI workflows | [docs.aixplain.com/concepts/assets/pipelines](https://docs.aixplain.com/concepts/assets/pipelines) | -| **Agents** | Deploy intelligent AI assistants | [docs.aixplain.com/concepts/assets/agents](https://docs.aixplain.com/concepts/assets/agents) | -| **Datasets** | Manage and process data | [docs.aixplain.com/concepts/assets/data/overview](https://docs.aixplain.com/concepts/assets/data/overview) | -| **Benchmarks** | Evaluate AI performance | [docs.aixplain.com/concepts/services/benchmark/benchmark-models](https://docs.aixplain.com/concepts/services/benchmark/benchmark-models) | -| **FineTuning** | Customize models with your data | [docs.aixplain.com/concepts/services/finetune/finetune-llm](https://docs.aixplain.com/concepts/services/finetune/finetune-llm) | +#Scrape tool +scrape_tool = ModelFactory.get("66f423426eb563fa213a3531") -## 📚 Documentation +# Create specialized agents +scraper_agent = AgentFactory.create( + name="Scraper Agent", + description="An agent that answers queries using website scraping.", + tasks=[scrape_task], + tools=[scrape_tool] +) -Comprehensive documentation and guides are available at **[docs.aixplain.com](https://docs.aixplain.com)**: +#Wiki tool +wiki_tool = ModelFactory.get("6633fd59821ee31dd914e232") -### 🎯 Getting Started -- [**Quick Start Guide**](https://docs.aixplain.com/getting-started/) - Get up and running in minutes -- [**API Key Setup**](https://docs.aixplain.com/getting-started/python) - Authentication and configuration -- [**Tutorials**](https://docs.aixplain.com/tutorials/) - Build your first AI application +wiki_agent = AgentFactory.create( + name="Wiki Agent", + description="An agent that answers queries using wikipedia.", + tasks=[wiki_task], + tools=[wiki_tool] +) -### 📖 Core Guides -- [**Discover**](https://docs.aixplain.com/concepts/assets/models) aiXplain’s ever-expanding catalog of 35,000+ ready-to-use AI models and utilize them. -- [**Benchmark**](https://docs.aixplain.com/concepts/services/benchmark/benchmark-models) AI systems by choosing models, datasets and metrics. -- [**Design**](https://docs.aixplain.com/concepts/assets/pipelines) their own custom pipelines and run them. -- [**FineTune**](https://docs.aixplain.com/concepts/services/finetune/finetune-llm) pre-trained models by tuning them using your data, enhancing their performance. +# Create the team agent to orchestrate them +team_agent = TeamAgentFactory.create( + name="Wiki and Web Team Agent", + description="You search using wiki or by web scraping URLs if appropriate.", + instructions="You take user queries and search them using wiki or by web scraping URLs if appropriate.", + agents=[scraper_agent, wiki_agent] +) +# Run and test the team agent +query = "Tell me about OpenAI. They have a website, https://openai.com/." +result = team_agent.run(query) -## 🛠️ Advanced Examples +print(result['data']['output']) -### Batch Processing -```python -# Process multiple inputs efficiently -inputs = ["text1", "text2", "text3"] -results = model.run_batch(inputs) +# Deploy the team agent for a permanent API endpoint +team_agent.deploy() ``` -## 🤝 Community & Support +--- + +## Security, compliance, and privacy + +aiXplain takes a governance-first approach to enterprise trust: -- **📖 Documentation**: [docs.aixplain.com](https://docs.aixplain.com) -- **💬 Discord Community**: [discord.gg/aixplain](https://discord.gg/aixplain) -- **🐛 Issues**: [GitHub Issues](https://github.com/aixplain/aiXplain/issues) -- **📧 Support**: support@aixplain.com -- **🔄 Release Notes**: [GitHub Releases](https://github.com/aixplain/aiXplain/releases) +- **SOC 2 compliant** — audited for security, confidentiality, and privacy. +- **No data used for training** — prompts, responses, and fine-tuned models stay private. +- **Data sovereignty** — full control with OnEdge and OnPrem options. +- **End-to-end encryption** — in transit (TLS 1.2+) and at rest. -## 🔗 Platform Links +Learn more at [aiXplain Security](https://aixplain.com/security/). -- **🏠 Platform Home**: [platform.aixplain.com](https://platform.aixplain.com) -- **🔍 Model Discovery**: [platform.aixplain.com/discovery/models](https://platform.aixplain.com/discovery/models) -- **📊 Datasets**: [platform.aixplain.com/discovery/datasets](https://platform.aixplain.com/discovery/datasets) -- **📏 Metrics**: [platform.aixplain.com/discovery/metrics](https://platform.aixplain.com/discovery/metrics) +--- -## 📄 License +## Pricing -This project is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details. +Start with our **Builder plan** — free credits at signup. -## 🏢 About aiXplain +- **Unlimited agents** — create and run without limits. +- **Pay as you go** — usage-based pricing only. +- **No idle costs** — pay nothing when agents aren't running. -aiXplain is the leading AI platform for building, deploying, and managing AI solutions at scale. We democratize AI by making cutting-edge models accessible to developers and enterprises worldwide. +Learn more at [aiXplain Pricing](https://aixplain.com/pricing/). --- -
- -**Ready to build the future with AI?** +## Community & support -[**Get Started →**](https://docs.aixplain.com/getting-started/) | [**Explore Models →**](https://platform.aixplain.com/discovery/models) | [**Join Community →**](https://discord.com/invite/T5dCmjRSYA) +- **Documentation:** [docs.aixplain.com](https://docs.aixplain.com) +- **Example agents**: [https://github.com/aixplain/cookbook](https://github.com/aixplain/cookbook) +- **Learn how to build agents**: [https://academy.aixplain.com/student-registration/](https://academy.aixplain.com/student-registration/) +- **Meet us in Discord:** [discord.gg/aixplain](https://www.google.com/search?q=https://discord.gg/aixplain) +- **Talk with our team:** [care@aixplain.com](mailto:care@aixplain.com) -
+--- + +## License + +This project is licensed under the Apache License 2.0. See the [`LICENSE`](LICENSE) file for details. diff --git a/aixplain/factories/team_agent_factory/inspector_factory.py b/aixplain/factories/team_agent_factory/inspector_factory.py index a6d38d4f..e7d88324 100644 --- a/aixplain/factories/team_agent_factory/inspector_factory.py +++ b/aixplain/factories/team_agent_factory/inspector_factory.py @@ -4,6 +4,8 @@ and monitor team agent operations. Inspectors can be created from existing models or using automatic configurations. +WARNING: This feature is currently in private beta. + Example: Create an inspector from a model with adaptive policy:: diff --git a/aixplain/modules/agent/__init__.py b/aixplain/modules/agent/__init__.py index 3ed4b04a..7973ac00 100644 --- a/aixplain/modules/agent/__init__.py +++ b/aixplain/modules/agent/__init__.py @@ -274,7 +274,7 @@ def run( wait_time: float = 0.5, content: Optional[Union[Dict[Text, Text], List[Text]]] = None, max_tokens: int = 4096, - max_iterations: int = 3, + max_iterations: int = 5, output_format: Optional[OutputFormat] = None, expected_output: Optional[Union[BaseModel, Text, dict]] = None, ) -> AgentResponse: @@ -307,6 +307,8 @@ def run( if history: validate_history(history) result_data = {} + if len(self.tasks) > 0: + max_iterations = 30 try: response = self.run_async( data=data, @@ -370,7 +372,7 @@ def run_async( parameters: Dict = {}, content: Optional[Union[Dict[Text, Text], List[Text]]] = None, max_tokens: int = 2048, - max_iterations: int = 10, + max_iterations: int = 5, output_format: Optional[OutputFormat] = None, expected_output: Optional[Union[BaseModel, Text, dict]] = None, ) -> AgentResponse: diff --git a/aixplain/modules/team_agent/inspector.py b/aixplain/modules/team_agent/inspector.py index 8cd12cc0..6e930fb9 100644 --- a/aixplain/modules/team_agent/inspector.py +++ b/aixplain/modules/team_agent/inspector.py @@ -1,5 +1,7 @@ """Pre-defined agent for inspecting the data flow within a team agent. +WARNING: This feature is currently in private beta. + Example usage: inspector = Inspector( diff --git a/docs/api-reference/python/aixplain/base/parameters.md b/docs/api-reference/python/aixplain/base/parameters.md new file mode 100644 index 00000000..14eaf90b --- /dev/null +++ b/docs/api-reference/python/aixplain/base/parameters.md @@ -0,0 +1,169 @@ +--- +sidebar_label: parameters +title: aixplain.base.parameters +--- + +### Parameter Objects + +```python +@dataclass +class Parameter() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/base/parameters.py#L6) + +A class representing a single parameter with its properties. + +**Attributes**: + +- `name` _str_ - The name of the parameter. +- `required` _bool_ - Whether the parameter is required or optional. +- `value` _Optional[Any]_ - The value of the parameter. Defaults to None. + +### BaseParameters Objects + +```python +class BaseParameters() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/base/parameters.py#L19) + +A base class for managing a collection of parameters. + +This class provides functionality to store, access, and manipulate parameters +in a structured way. Parameters can be accessed using attribute syntax or +dictionary-style access. + +**Attributes**: + +- `parameters` _Dict[str, Parameter]_ - Dictionary storing Parameter objects. + +#### \_\_init\_\_ + +```python +def __init__() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/base/parameters.py#L29) + +Initialize the BaseParameters class. + +The initialization creates an empty dictionary to store parameters. + +#### get\_parameter + +```python +def get_parameter(name: str) -> Optional[Parameter] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/base/parameters.py#L36) + +Get a parameter by name. + +**Arguments**: + +- `name` _str_ - Name of the parameter + + +**Returns**: + +- `Optional[Parameter]` - Parameter object if found, None otherwise + +#### to\_dict + +```python +def to_dict() -> Dict[str, Dict[str, Any]] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/base/parameters.py#L47) + +Convert parameters back to dictionary format. + +**Returns**: + + Dict[str, Dict[str, Any]]: Dictionary representation of parameters + +#### to\_list + +```python +def to_list() -> List[str] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/base/parameters.py#L55) + +Convert parameters to a list format. + +This method creates a list of dictionaries containing the name and value +of each parameter that has a value set. + +**Returns**: + +- `List[str]` - A list of dictionaries, each containing 'name' and 'value' + keys for parameters that have values set. + +#### \_\_str\_\_ + +```python +def __str__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/base/parameters.py#L67) + +Create a pretty string representation of the parameters. + +**Returns**: + +- `str` - Formatted string showing all parameters + +#### \_\_setattr\_\_ + +```python +def __setattr__(name: str, value: Any) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/base/parameters.py#L84) + +Allow setting parameters using attribute syntax. + +This special method enables setting parameter values using attribute syntax +(e.g., params.text = "Hello"). It only works for parameters that have been +previously defined. + +**Arguments**: + +- `name` _str_ - Name of the parameter to set. +- `value` _Any_ - Value to assign to the parameter. + + +**Raises**: + +- `AttributeError` - If attempting to set a parameter that hasn't been defined. + +#### \_\_getattr\_\_ + +```python +def __getattr__(name: str) -> Any +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/base/parameters.py#L107) + +Allow getting parameter values using attribute syntax. + +This special method enables accessing parameter values using attribute syntax +(e.g., params.text). It only works for parameters that have been previously +defined. + +**Arguments**: + +- `name` _str_ - Name of the parameter to access. + + +**Returns**: + +- `Any` - The value of the requested parameter. + + +**Raises**: + +- `AttributeError` - If attempting to access a parameter that hasn't been defined. + diff --git a/docs/api-reference/python/aixplain/cli_groups.md b/docs/api-reference/python/aixplain/cli_groups.md new file mode 100644 index 00000000..f697640e --- /dev/null +++ b/docs/api-reference/python/aixplain/cli_groups.md @@ -0,0 +1,26 @@ +--- +sidebar_label: cli_groups +title: aixplain.cli_groups +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Michael Lam +Date: September 18th 2023 +Description: + CLI Runner + diff --git a/docs/api-reference/python/aixplain/decorators/api_key_checker.md b/docs/api-reference/python/aixplain/decorators/api_key_checker.md new file mode 100644 index 00000000..5c6e27c3 --- /dev/null +++ b/docs/api-reference/python/aixplain/decorators/api_key_checker.md @@ -0,0 +1,40 @@ +--- +sidebar_label: api_key_checker +title: aixplain.decorators.api_key_checker +--- + +#### check\_api\_key + +```python +def check_api_key(method) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/decorators/api_key_checker.py#L4) + +Decorator to verify that an API key is set before executing the method. + +This decorator checks if either TEAM_API_KEY or AIXPLAIN_API_KEY is set in the +configuration. If neither key is set, it raises an exception. + +**Arguments**: + +- `method` _callable_ - The method to be decorated. + + +**Returns**: + +- `callable` - The wrapped method that includes API key verification. + + +**Raises**: + +- `Exception` - If neither TEAM_API_KEY nor AIXPLAIN_API_KEY is set. + + +**Example**: + + @check_api_key + def my_api_method(): + # Method implementation + pass + diff --git a/docs/api-reference/python/aixplain/decorators/init.md b/docs/api-reference/python/aixplain/decorators/init.md new file mode 100644 index 00000000..a9eee264 --- /dev/null +++ b/docs/api-reference/python/aixplain/decorators/init.md @@ -0,0 +1,6 @@ +--- +draft: true +sidebar_label: decorators +title: aixplain.decorators +--- + diff --git a/docs/api-reference/python/aixplain/enums/asset_status.md b/docs/api-reference/python/aixplain/enums/asset_status.md new file mode 100644 index 00000000..8c91a394 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/asset_status.md @@ -0,0 +1,61 @@ +--- +sidebar_label: asset_status +title: aixplain.enums.asset_status +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: February 21st 2024 +Description: + Asset Enum + +### AssetStatus Objects + +```python +class AssetStatus(Text, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/asset_status.py#L28) + +Enumeration of possible status values for an asset in the aiXplain system. + +This enum defines all possible states that an asset can be in throughout its lifecycle, +from creation to deletion. Each enum value corresponds to a specific state in the +asset's lifecycle. + +**Attributes**: + +- `DRAFT` _str_ - Initial state for a newly created asset. +- `HIDDEN` _str_ - Asset is hidden from public view. +- `SCHEDULED` _str_ - Asset is scheduled for processing. +- `ONBOARDING` _str_ - Asset is in the process of being onboarded. +- `ONBOARDED` _str_ - Asset has been successfully onboarded. +- `PENDING` _str_ - Asset is waiting for processing. +- `FAILED` _str_ - Asset processing has failed. +- `TRAINING` _str_ - Asset is currently in training. +- `REJECTED` _str_ - Asset has been rejected. +- `ENABLING` _str_ - Asset is in the process of being enabled. +- `HIDDEN`0 _str_ - Asset is in the process of being deleted. +- `HIDDEN`1 _str_ - Asset has been disabled. +- `HIDDEN`2 _str_ - Asset has been deleted. +- `HIDDEN`3 _str_ - Asset is currently being processed. +- `HIDDEN`4 _str_ - Asset has completed processing. +- `HIDDEN`5 _str_ - Asset operation is being canceled. +- `HIDDEN`6 _str_ - Asset operation has been canceled. +- `HIDDEN`7 _str_ - Draft state that has been deprecated. + diff --git a/docs/api-reference/python/aixplain/enums/code_interpreter.md b/docs/api-reference/python/aixplain/enums/code_interpreter.md new file mode 100644 index 00000000..80f98654 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/code_interpreter.md @@ -0,0 +1,37 @@ +--- +sidebar_label: code_interpreter +title: aixplain.enums.code_interpreter +--- + +### CodeInterpreterModel Objects + +```python +class CodeInterpreterModel(str, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/code_interpreter.py#L4) + +Enumeration of available Code Interpreter model identifiers. + +This enum defines the unique identifiers for different code interpreter models +available in the system. Each value represents a specific model's ID that can +be used for code interpretation tasks. + +**Attributes**: + +- `PYTHON_AZURE` _str_ - Model ID for the Python code interpreter running on Azure. + +#### \_\_str\_\_ + +```python +def __str__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/code_interpreter.py#L17) + +Return the string representation of the model ID. + +**Returns**: + +- `str` - The model ID value as a string. + diff --git a/docs/api-reference/python/aixplain/enums/data_split.md b/docs/api-reference/python/aixplain/enums/data_split.md new file mode 100644 index 00000000..ddc9fa76 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/data_split.md @@ -0,0 +1,45 @@ +--- +sidebar_label: data_split +title: aixplain.enums.data_split +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 20th 2023 +Description: + Data Split Enum + +### DataSplit Objects + +```python +class DataSplit(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/data_split.py#L27) + +Enumeration of dataset split types. + +This enum defines the standard dataset split types used for machine learning tasks, +including training, validation, and testing splits. + +**Attributes**: + +- `TRAIN` _str_ - Training dataset split used for model training. +- `VALIDATION` _str_ - Validation dataset split used for model tuning. +- `TEST` _str_ - Test dataset split used for final model evaluation. + diff --git a/docs/api-reference/python/aixplain/enums/data_subtype.md b/docs/api-reference/python/aixplain/enums/data_subtype.md new file mode 100644 index 00000000..3406428c --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/data_subtype.md @@ -0,0 +1,64 @@ +--- +sidebar_label: data_subtype +title: aixplain.enums.data_subtype +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: May 3rd 2023 +Description: + Data Subtype Enum + +### DataSubtype Objects + +```python +class DataSubtype(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/data_subtype.py#L27) + +Enumeration of data subtypes for categorizing and organizing data. + +This enum defines various subtypes that can be used to further categorize +data points within the system, particularly useful for demographic and +content-based categorization. + +**Attributes**: + +- `AGE` _str_ - Age category subtype. +- `GENDER` _str_ - Gender category subtype. +- `INTERVAL` _str_ - Time interval subtype. +- `OTHER` _str_ - Miscellaneous/other subtype. +- `RACE` _str_ - Race/ethnicity category subtype. +- `SPLIT` _str_ - Data split category subtype. +- `TOPIC` _str_ - Content topic subtype. + +#### \_\_str\_\_ + +```python +def __str__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/data_subtype.py#L51) + +Return the string representation of the data subtype. + +**Returns**: + +- `str` - The data subtype value as a string. + diff --git a/docs/api-reference/python/aixplain/enums/data_type.md b/docs/api-reference/python/aixplain/enums/data_type.md new file mode 100644 index 00000000..ac82d907 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/data_type.md @@ -0,0 +1,67 @@ +--- +sidebar_label: data_type +title: aixplain.enums.data_type +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 20th 2023 +Description: + Data Type Enum + +### DataType Objects + +```python +class DataType(str, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/data_type.py#L27) + +Enumeration of supported data types in the aiXplain system. + +This enum defines all the data types that can be processed by the system, +including various media types and basic data types. + +**Attributes**: + +- `AUDIO` _str_ - Audio data type. +- `FLOAT` _str_ - Floating-point number data type. +- `IMAGE` _str_ - Image data type. +- `INTEGER` _str_ - Integer number data type. +- `LABEL` _str_ - Label/category data type. +- `TENSOR` _str_ - Tensor/multi-dimensional array data type. +- `TEXT` _str_ - Text data type. +- `VIDEO` _str_ - Video data type. +- `EMBEDDING` _str_ - Vector embedding data type. +- `NUMBER` _str_ - Generic number data type. +- `FLOAT`0 _str_ - Boolean data type. + +#### \_\_str\_\_ + +```python +def __str__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/data_type.py#L58) + +Return the string representation of the data type. + +**Returns**: + +- `str` - The data type value as a string. + diff --git a/docs/api-reference/python/aixplain/enums/database_source.md b/docs/api-reference/python/aixplain/enums/database_source.md new file mode 100644 index 00000000..557cd161 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/database_source.md @@ -0,0 +1,65 @@ +--- +sidebar_label: database_source +title: aixplain.enums.database_source +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Lucas Pavanelli and Thiago Castro Ferreira and Ahmet Gunduz +Date: March 7th 2025 +Description: + Database Source Type Enum + +### DatabaseSourceType Objects + +```python +class DatabaseSourceType(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/database_source.py#L27) + +Enumeration of supported database source types. + +This enum defines the different types of database sources that can be used +for data storage and retrieval in the system. + +**Attributes**: + +- `POSTGRESQL` _str_ - PostgreSQL database source type. +- `SQLITE` _str_ - SQLite database source type. +- `CSV` _str_ - CSV file source type. + +#### from\_string + +```python +@classmethod +def from_string(cls, source_type: str) -> "DatabaseSourceType" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/database_source.py#L44) + +Convert string to DatabaseSourceType enum + +**Arguments**: + +- `source_type` _str_ - Source type string + + +**Returns**: + +- `DatabaseSourceType` - Corresponding enum value + diff --git a/docs/api-reference/python/aixplain/enums/embedding_model.md b/docs/api-reference/python/aixplain/enums/embedding_model.md new file mode 100644 index 00000000..f21253e6 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/embedding_model.md @@ -0,0 +1,56 @@ +--- +sidebar_label: embedding_model +title: aixplain.enums.embedding_model +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +Author: aiXplain team +Date: February 17th 2025 +Description: + Embedding Model Enum + +### EmbeddingModel Objects + +```python +class EmbeddingModel(str, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/embedding_model.py#L23) + +Enumeration of available embedding models in the aiXplain system. + +This enum defines the unique identifiers for different embedding models that can +be used to generate vector representations of data. + +**Attributes**: + +- `OPENAI_ADA002` _str_ - OpenAI's Ada-002 text embedding model ID. +- `JINA_CLIP_V2_MULTIMODAL` _str_ - Jina CLIP v2 multimodal embedding model ID. +- `MULTILINGUAL_E5_LARGE` _str_ - Multilingual E5 Large text embedding model ID. +- `BGE_M3` _str_ - BGE-M3 embedding model ID. + +#### \_\_str\_\_ + +```python +def __str__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/embedding_model.py#L40) + +Return the string representation of the embedding model ID. + +**Returns**: + +- `str` - The model ID value as a string. + diff --git a/docs/api-reference/python/aixplain/enums/error_handler.md b/docs/api-reference/python/aixplain/enums/error_handler.md new file mode 100644 index 00000000..a15929aa --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/error_handler.md @@ -0,0 +1,41 @@ +--- +sidebar_label: error_handler +title: aixplain.enums.error_handler +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: May 26th 2023 +Description: + Error Handler Enum + +### ErrorHandler Objects + +```python +class ErrorHandler(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/error_handler.py#L27) + +Enumeration class defining different error handler strategies. + +**Attributes**: + +- `SKIP` _str_ - skip failed rows. +- `FAIL` _str_ - raise an exception. + diff --git a/docs/api-reference/python/aixplain/enums/file_type.md b/docs/api-reference/python/aixplain/enums/file_type.md new file mode 100644 index 00000000..551fca08 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/file_type.md @@ -0,0 +1,58 @@ +--- +sidebar_label: file_type +title: aixplain.enums.file_type +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 20th 2023 +Description: + File Type Enum + +### FileType Objects + +```python +class FileType(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/file_type.py#L27) + +Enumeration of supported file types in the aiXplain system. + +This enum defines the file extensions for various file formats that can be +processed by the system, including document, audio, image, and video formats. + +**Attributes**: + +- `CSV` _str_ - Comma-separated values file (.csv). +- `JSON` _str_ - JSON document file (.json). +- `TXT` _str_ - Plain text file (.txt). +- `XML` _str_ - XML document file (.xml). +- `FLAC` _str_ - Free Lossless Audio Codec file (.flac). +- `MP3` _str_ - MP3 audio file (.mp3). +- `WAV` _str_ - Waveform audio file (.wav). +- `JPEG` _str_ - JPEG image file (.jpeg). +- `PNG` _str_ - Portable Network Graphics file (.png). +- `JPG` _str_ - JPEG image file (.jpg). +- `JSON`0 _str_ - Graphics Interchange Format file (.gif). +- `JSON`1 _str_ - WebP image file (.webp). +- `JSON`2 _str_ - Audio Video Interleave file (.avi). +- `JSON`3 _str_ - MPEG-4 video file (.mp4). +- `JSON`4 _str_ - QuickTime movie file (.mov). +- `JSON`5 _str_ - MPEG-4 video file (.mpeg4). + diff --git a/docs/api-reference/python/aixplain/enums/function.md b/docs/api-reference/python/aixplain/enums/function.md new file mode 100644 index 00000000..8944660b --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/function.md @@ -0,0 +1,132 @@ +--- +sidebar_label: function +title: aixplain.enums.function +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 20th 2023 +Description: + Function Enum + +### FunctionMetadata Objects + +```python +@dataclass +class FunctionMetadata() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/function.py#L40) + +Metadata container for function information. + +This class holds metadata about a function including its identifier, name, +description, parameters, outputs, and additional metadata. + +**Attributes**: + +- `id` _str_ - ID of the function. +- `name` _str_ - Name of the function. +- `description` _Optional[str]_ - Description of what the function does. +- `params` _List[Dict[str, Any]]_ - List of parameter specifications. +- `output` _List[Dict[str, Any]]_ - List of output specifications. +- `metadata` _Dict[str, Any]_ - Additional metadata about the function. + +#### to\_dict + +```python +def to_dict() -> dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/function.py#L61) + +Convert the function metadata to a dictionary. + +**Returns**: + +- `dict` - Dictionary representation of the function metadata. + +#### from\_dict + +```python +@classmethod +def from_dict(cls, data: dict) -> "FunctionMetadata" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/function.py#L77) + +Create a FunctionMetadata instance from a dictionary. + +**Arguments**: + +- `data` _dict_ - Dictionary containing function metadata. + + +**Returns**: + +- `FunctionMetadata` - New instance created from the dictionary data. + +#### load\_functions + +```python +def load_functions() -> Tuple[Enum, Dict] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/function.py#L96) + +Load function definitions from the backend or cache. + +This function attempts to load function definitions from the cache first. +If the cache is invalid or doesn't exist, it fetches the data from the +backend API. + +**Returns**: + + Tuple[Function, Dict]: A tuple containing: + - Function: Dynamically created Function enum class + - Dict: Dictionary mapping function IDs to their input/output specifications + + +**Raises**: + +- `Exception` - If functions cannot be loaded due to invalid API key or other errors. + +### FunctionParameters Objects + +```python +class FunctionParameters(BaseParameters) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/function.py#L203) + +Class to store and manage function parameters + +#### \_\_init\_\_ + +```python +def __init__(input_params: Dict) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/function.py#L206) + +Initialize FunctionParameters with input parameters + +**Arguments**: + +- `input_params` _Dict_ - Dictionary of input parameters + diff --git a/docs/api-reference/python/aixplain/enums/function_type.md b/docs/api-reference/python/aixplain/enums/function_type.md new file mode 100644 index 00000000..a449c7cf --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/function_type.md @@ -0,0 +1,53 @@ +--- +sidebar_label: function_type +title: aixplain.enums.function_type +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: May 22th 2025 +Description: + Function Type Enum + +### FunctionType Objects + +```python +class FunctionType(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/function_type.py#L27) + +Enumeration of function types in the aiXplain system. + +This enum defines the different types of functions and services available +in the system, including AI models, data processing utilities, and +integration components. + +**Attributes**: + +- `AI` _str_ - Artificial Intelligence function type. +- `SEGMENTOR` _str_ - Data segmentation function type. +- `RECONSTRUCTOR` _str_ - Data reconstruction function type. +- `UTILITY` _str_ - Utility function type. +- `METRIC` _str_ - Metric/evaluation function type. +- `SEARCH` _str_ - Search function type. +- `INTEGRATION` _str_ - Integration connector function type. # i.e. slack +- `CONNECTION` _str_ - Connection function type. # slack - action +- `MCP_CONNECTION` _str_ - MCP connection function type. +- `MCPSERVER` _str_ - MCP server is for on-prem solution. It should be treated like a model. # ONPREM_MCP_MODEL + diff --git a/docs/api-reference/python/aixplain/enums/index_stores.md b/docs/api-reference/python/aixplain/enums/index_stores.md new file mode 100644 index 00000000..0b83a21f --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/index_stores.md @@ -0,0 +1,53 @@ +--- +sidebar_label: index_stores +title: aixplain.enums.index_stores +--- + +### IndexStores Objects + +```python +class IndexStores(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/index_stores.py#L4) + +Enumeration of available index store providers in the aiXplain system. + +This enum defines the different index store providers that can be used for +storing and retrieving indexed data, along with their identifiers. + +**Attributes**: + +- `AIR` _dict_ - AIR index store configuration with name and ID. +- `VECTARA` _dict_ - Vectara index store configuration with name and ID. +- `GRAPHRAG` _dict_ - GraphRAG index store configuration with name and ID. +- `ZERO_ENTROPY` _dict_ - Zero Entropy index store configuration with name and ID. + +#### \_\_str\_\_ + +```python +def __str__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/index_stores.py#L21) + +Return the name of the index store. + +**Returns**: + +- `str` - The name value from the index store configuration. + +#### get\_model\_id + +```python +def get_model_id() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/index_stores.py#L29) + +Return the model ID of the index store. + +**Returns**: + +- `str` - The ID value from the index store configuration. + diff --git a/docs/api-reference/python/aixplain/enums/init.md b/docs/api-reference/python/aixplain/enums/init.md new file mode 100644 index 00000000..1c2c4064 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/init.md @@ -0,0 +1,6 @@ +--- +draft: true +sidebar_label: enums +title: aixplain.enums +--- + diff --git a/docs/api-reference/python/aixplain/enums/language.md b/docs/api-reference/python/aixplain/enums/language.md new file mode 100644 index 00000000..723ded28 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/language.md @@ -0,0 +1,106 @@ +--- +sidebar_label: language +title: aixplain.enums.language +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 22th 2023 +Description: + Language Enum + +### LanguageMetadata Objects + +```python +@dataclass +class LanguageMetadata() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/language.py#L34) + +Metadata container for language information. + +This class holds metadata about a language including its identifier, value, +label, dialects, and supported scripts. + +**Attributes**: + +- `id` _str_ - ID of the language. +- `value` _str_ - Language code or value. +- `label` _str_ - Label for the language. +- `dialects` _List[Dict[str, str]]_ - List of dialect specifications. +- `scripts` _List[Any]_ - List of supported scripts for the language. + +#### to\_dict + +```python +def to_dict() -> dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/language.py#L53) + +Convert the language metadata to a dictionary. + +**Returns**: + +- `dict` - Dictionary representation of the language metadata. + +#### from\_dict + +```python +@classmethod +def from_dict(cls, data: dict) -> "LanguageMetadata" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/language.py#L68) + +Create a LanguageMetadata instance from a dictionary. + +**Arguments**: + +- `data` _dict_ - Dictionary containing language metadata. + + +**Returns**: + +- `LanguageMetadata` - New instance created from the dictionary data. + +#### load\_languages + +```python +def load_languages() -> Enum +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/language.py#L85) + +Load language definitions from the backend or cache. + +This function attempts to load language definitions from the cache first. +If the cache is invalid or doesn't exist, it fetches the data from the +backend API. It creates a dynamic Enum class containing all available +languages and their dialects. + +**Returns**: + +- `Enum` - Dynamically created Language enum class with language codes and dialects. + + +**Raises**: + +- `Exception` - If languages cannot be loaded due to invalid API key or other errors. + diff --git a/docs/api-reference/python/aixplain/enums/license.md b/docs/api-reference/python/aixplain/enums/license.md new file mode 100644 index 00000000..94a8a1a5 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/license.md @@ -0,0 +1,106 @@ +--- +sidebar_label: license +title: aixplain.enums.license +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 20th 2023 +Description: + License Enum + +### LicenseMetadata Objects + +```python +@dataclass +class LicenseMetadata() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/license.py#L34) + +Metadata container for license information. + +This class holds metadata about a license including its identifier, name, +description, URL, and custom URL settings. + +**Attributes**: + +- `id` _str_ - ID of the license. +- `name` _str_ - Name of the license. +- `description` _str_ - Description of the license terms. +- `url` _str_ - URL to the license text or details. +- `allowCustomUrl` _bool_ - Whether custom URLs are allowed for this license. + +#### to\_dict + +```python +def to_dict() -> dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/license.py#L53) + +Convert the license metadata to a dictionary. + +**Returns**: + +- `dict` - Dictionary representation of the license metadata. + +#### from\_dict + +```python +@classmethod +def from_dict(cls, data: dict) -> "LicenseMetadata" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/license.py#L68) + +Create a LicenseMetadata instance from a dictionary. + +**Arguments**: + +- `data` _dict_ - Dictionary containing license metadata. + + +**Returns**: + +- `LicenseMetadata` - New instance created from the dictionary data. + +#### load\_licenses + +```python +def load_licenses() -> Enum +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/license.py#L86) + +Load license definitions from the backend or cache. + +This function attempts to load license definitions from the cache first. +If the cache is invalid or doesn't exist, it fetches the data from the +backend API. It creates a dynamic Enum class containing all available +licenses. + +**Returns**: + +- `Enum` - Dynamically created License enum class with license identifiers. + + +**Raises**: + +- `Exception` - If licenses cannot be loaded due to invalid API key or other errors. + diff --git a/docs/api-reference/python/aixplain/enums/onboard_status.md b/docs/api-reference/python/aixplain/enums/onboard_status.md new file mode 100644 index 00000000..b67c5570 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/onboard_status.md @@ -0,0 +1,46 @@ +--- +sidebar_label: onboard_status +title: aixplain.enums.onboard_status +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 22th 2023 +Description: + Onboard Status Enum + +### OnboardStatus Objects + +```python +class OnboardStatus(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/onboard_status.py#L27) + +Enumeration of possible onboarding status values. + +This enum defines all possible states that an onboarding process can be in, +from initial onboarding to completed or failed states. + +**Attributes**: + +- `ONBOARDING` _str_ - Initial onboarding state. +- `ONBOARDED` _str_ - Successful onboarding state. +- `FAILED` _str_ - Failed onboarding state. +- `DELETED` _str_ - Deleted onboarding state. + diff --git a/docs/api-reference/python/aixplain/enums/ownership_type.md b/docs/api-reference/python/aixplain/enums/ownership_type.md new file mode 100644 index 00000000..0706b199 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/ownership_type.md @@ -0,0 +1,58 @@ +--- +sidebar_label: ownership_type +title: aixplain.enums.ownership_type +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: November 22nd 2023 +Description: + Asset Ownership Type + +### OwnershipType Objects + +```python +class OwnershipType(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/ownership_type.py#L27) + +Enumeration of possible ownership types. + +This enum defines the different types of ownership that can be associated with +an asset or resource, including subscribed and owned ownership. + +**Attributes**: + +- `SUBSCRIBED` _str_ - Subscribed ownership type. +- `OWNED` _str_ - Owned ownership type. + +#### \_\_str\_\_ + +```python +def __str__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/ownership_type.py#L40) + +Return the string representation of the ownership type. + +**Returns**: + +- `str` - The ownership type value as a string. + diff --git a/docs/api-reference/python/aixplain/enums/privacy.md b/docs/api-reference/python/aixplain/enums/privacy.md new file mode 100644 index 00000000..7d0568f8 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/privacy.md @@ -0,0 +1,45 @@ +--- +sidebar_label: privacy +title: aixplain.enums.privacy +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 20th 2023 +Description: + Privacy Enum + +### Privacy Objects + +```python +class Privacy(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/privacy.py#L27) + +Enumeration of possible privacy levels. + +This enum defines the different levels of privacy that can be associated with +an asset or resource, including public, private, and restricted privacy levels. + +**Attributes**: + +- `PUBLIC` _str_ - Public privacy level. +- `PRIVATE` _str_ - Private privacy level. +- `RESTRICTED` _str_ - Restricted privacy level. + diff --git a/docs/api-reference/python/aixplain/enums/response_status.md b/docs/api-reference/python/aixplain/enums/response_status.md new file mode 100644 index 00000000..089bb724 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/response_status.md @@ -0,0 +1,59 @@ +--- +sidebar_label: response_status +title: aixplain.enums.response_status +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: February 21st 2024 +Description: + Asset Enum + +### ResponseStatus Objects + +```python +class ResponseStatus(Text, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/response_status.py#L28) + +Enumeration of possible response status values. + +This enum defines the different statuses that a response can be in, including +in progress, success, and failure. + +**Attributes**: + +- `IN_PROGRESS` _str_ - Response is in progress. +- `SUCCESS` _str_ - Response was successful. +- `FAILED` _str_ - Response failed. + +#### \_\_str\_\_ + +```python +def __str__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/response_status.py#L43) + +Return the string representation of the response status. + +**Returns**: + +- `str` - The response status value as a string. + diff --git a/docs/api-reference/python/aixplain/enums/sort_by.md b/docs/api-reference/python/aixplain/enums/sort_by.md new file mode 100644 index 00000000..2abf8380 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/sort_by.md @@ -0,0 +1,45 @@ +--- +sidebar_label: sort_by +title: aixplain.enums.sort_by +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 20th 2023 +Description: + Sort By Enum + +### SortBy Objects + +```python +class SortBy(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/sort_by.py#L27) + +Enumeration of possible sorting criteria. + +This enum defines the different criteria that can be used to sort assets, +including creation date, price, and popularity. + +**Attributes**: + +- `CREATION_DATE` _str_ - Sort by creation date. +- `PRICE` _str_ - Sort by normalized price. +- `POPULARITY` _str_ - Sort by total number of subscriptions. + diff --git a/docs/api-reference/python/aixplain/enums/sort_order.md b/docs/api-reference/python/aixplain/enums/sort_order.md new file mode 100644 index 00000000..459ef0bf --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/sort_order.md @@ -0,0 +1,44 @@ +--- +sidebar_label: sort_order +title: aixplain.enums.sort_order +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 20th 2023 +Description: + Sort By Enum + +### SortOrder Objects + +```python +class SortOrder(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/sort_order.py#L27) + +Enumeration of possible sorting orders. + +This enum defines the different directions that can be used to sort assets, +including ascending and descending order. + +**Attributes**: + +- `ASCENDING` _int_ - Sort in ascending order. +- `DESCENDING` _int_ - Sort in descending order. + diff --git a/docs/api-reference/python/aixplain/enums/splitting_options.md b/docs/api-reference/python/aixplain/enums/splitting_options.md new file mode 100644 index 00000000..2a6a7d7f --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/splitting_options.md @@ -0,0 +1,61 @@ +--- +sidebar_label: splitting_options +title: aixplain.enums.splitting_options +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: May 30th 2025 +Description: + Splitting Options Enum + +### SplittingOptions Objects + +```python +class SplittingOptions(str, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/splitting_options.py#L27) + +Enumeration of possible splitting options. + +This enum defines the different ways that text can be split into chunks, +including by word, sentence, passage, page, and line. + +**Attributes**: + +- `WORD` _str_ - Split by word. +- `SENTENCE` _str_ - Split by sentence. +- `PASSAGE` _str_ - Split by passage. +- `PAGE` _str_ - Split by page. +- `LINE` _str_ - Split by line. + +#### \_\_str\_\_ + +```python +def __str__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/splitting_options.py#L46) + +Return the string representation of the splitting option. + +**Returns**: + +- `str` - The splitting option value as a string. + diff --git a/docs/api-reference/python/aixplain/enums/status.md b/docs/api-reference/python/aixplain/enums/status.md new file mode 100644 index 00000000..f7330c17 --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/status.md @@ -0,0 +1,24 @@ +--- +sidebar_label: status +title: aixplain.enums.status +--- + +### Status Objects + +```python +class Status(Text, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/status.py#L5) + +Enumeration of possible status values. + +This enum defines the different statuses that a task or operation can be in, +including failed, in progress, and success. + +**Attributes**: + +- `FAILED` _str_ - Task failed. +- `IN_PROGRESS` _str_ - Task is in progress. +- `SUCCESS` _str_ - Task was successful. + diff --git a/docs/api-reference/python/aixplain/enums/storage_type.md b/docs/api-reference/python/aixplain/enums/storage_type.md new file mode 100644 index 00000000..052df52f --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/storage_type.md @@ -0,0 +1,59 @@ +--- +sidebar_label: storage_type +title: aixplain.enums.storage_type +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 20th 2023 +Description: + Storage Type Enum + +### StorageType Objects + +```python +class StorageType(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/storage_type.py#L27) + +Enumeration of possible storage types. + +This enum defines the different types of storage that can be used to store +assets, including text, URL, and file. + +**Attributes**: + +- `TEXT` _str_ - Text storage type. +- `URL` _str_ - URL storage type. +- `FILE` _str_ - File storage type. + +#### \_\_str\_\_ + +```python +def __str__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/storage_type.py#L42) + +Return the string representation of the storage type. + +**Returns**: + +- `str` - The storage type value as a string. + diff --git a/docs/api-reference/python/aixplain/enums/supplier.md b/docs/api-reference/python/aixplain/enums/supplier.md new file mode 100644 index 00000000..70aa8ebd --- /dev/null +++ b/docs/api-reference/python/aixplain/enums/supplier.md @@ -0,0 +1,72 @@ +--- +sidebar_label: supplier +title: aixplain.enums.supplier +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: September 25th 2023 +Description: + Supplier Enum + +#### clean\_name + +```python +def clean_name(name: str) -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/supplier.py#L33) + +Clean a supplier name by replacing spaces and special characters with underscores. + +This function takes a supplier name and performs the following transformations: +1. Replaces spaces and hyphens with underscores. +2. Removes any non-alphanumeric characters. +3. Removes any leading numbers. + +**Arguments**: + +- `name` _str_ - The supplier name to clean. + + +**Returns**: + +- `str` - The cleaned supplier name. + +#### load\_suppliers + +```python +def load_suppliers() -> Enum +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/enums/supplier.py#L53) + +Load suppliers from the backend or cache. + +This function fetches supplier information from the backend API and creates +an Enum class with supplier names as keys. + +**Returns**: + +- `Enum` - An Enum class with supplier names as keys. + + +**Raises**: + +- `Exception` - If suppliers cannot be loaded due to invalid API key or other errors. + diff --git a/docs/api-reference/python/aixplain/exceptions/init.md b/docs/api-reference/python/aixplain/exceptions/init.md new file mode 100644 index 00000000..67f62e9d --- /dev/null +++ b/docs/api-reference/python/aixplain/exceptions/init.md @@ -0,0 +1,32 @@ +--- +sidebar_label: exceptions +title: aixplain.exceptions +--- + +Error message registry for aiXplain SDK. + +This module maintains a centralized registry of error messages used throughout the aiXplain ecosystem. +It allows developers to look up existing error messages and reuse them instead of creating new ones. + +#### get\_error\_from\_status\_code + +```python +def get_error_from_status_code(status_code: int, + error_details: str = None + ) -> AixplainBaseException +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/__init__.py#L21) + +Map HTTP status codes to appropriate exception types. + +**Arguments**: + +- `status_code` _int_ - The HTTP status code to map. +- `default_message` _str, optional_ - The default message to use if no specific message is available. + + +**Returns**: + +- `AixplainBaseException` - An exception of the appropriate type. + diff --git a/docs/api-reference/python/aixplain/exceptions/types.md b/docs/api-reference/python/aixplain/exceptions/types.md new file mode 100644 index 00000000..ee386c4e --- /dev/null +++ b/docs/api-reference/python/aixplain/exceptions/types.md @@ -0,0 +1,318 @@ +--- +sidebar_label: types +title: aixplain.exceptions.types +--- + +### ErrorSeverity Objects + +```python +class ErrorSeverity(str, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L5) + +Enumeration of error severity levels in the aiXplain system. + +This enum defines the different levels of severity that can be assigned to +errors, ranging from informational messages to critical system errors. + +**Attributes**: + +- `INFO` _str_ - Informational message, not an actual error. +- `WARNING` _str_ - Warning that doesn't prevent operation completion. +- `ERROR` _str_ - Error condition that prevents operation completion. +- `CRITICAL` _str_ - Severe error that might affect system stability. + +#### INFO + +Informational, not an error + +#### WARNING + +Warning, operation can continue + +#### ERROR + +Error, operation cannot continue + +#### CRITICAL + +System stability might be compromised + +### ErrorCategory Objects + +```python +class ErrorCategory(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L24) + +Enumeration of error categories in the aiXplain system. + +This enum defines the different domains or areas where errors can occur, +helping to classify and organize error handling. + +**Attributes**: + +- `AUTHENTICATION` _str_ - Authentication and authorization errors. +- `VALIDATION` _str_ - Input validation errors. +- `RESOURCE` _str_ - Resource availability and access errors. +- `BILLING` _str_ - Billing and payment-related errors. +- `SUPPLIER` _str_ - External supplier and third-party service errors. +- `NETWORK` _str_ - Network connectivity errors. +- `SERVICE` _str_ - Service availability errors. +- `INTERNAL` _str_ - Internal system errors. +- `AGENT` _str_ - Agent-specific errors. +- `UNKNOWN` _str_ - Uncategorized or unclassified errors. + +#### AUTHENTICATION + +API keys, permissions + +#### VALIDATION + +Input validation + +#### RESOURCE + +Resource availability + +#### BILLING + +Credits, payment + +#### SUPPLIER + +External supplier issues + +#### NETWORK + +Network connectivity + +#### SERVICE + +Service availability + +#### INTERNAL + +Internal system errors + +#### AGENT + +Agent-specific errors + +#### UNKNOWN + +Uncategorized errors + +### ErrorCode Objects + +```python +class ErrorCode(str, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L55) + +Standard error codes for aiXplain exceptions. + +The format is AX-<CATEGORY>-<ID>, where <CATEGORY> is a short identifier +derived from the ErrorCategory (e.g., AUTH, VAL, RES) and <ID> is a +unique sequential number within that category, starting from 1000. + +How to Add a New Error Code: +1. Identify the appropriate `ErrorCategory` for the new error. +2. Determine the next available sequential ID within that category. + For example, if `AX-AUTH-1000` exists, the next authentication-specific + error could be `AX-AUTH-1001`. +3. Define the new enum member using the format `AX--`. + Use a concise abbreviation for the category (e.g., AUTH, VAL, RES, BIL, + SUP, NET, SVC, INT). +4. Assign the string value (e.g., `"AX-AUTH-1001"`). +5. Add a clear docstring explaining the specific condition that triggers + this error code. +6. (Optional but recommended) Consider creating a more specific exception + class inheriting from the corresponding category exception (e.g., + `class InvalidApiKeyError(AuthenticationError): ...`) and assign the + new error code to it. + +#### AX\_AUTH\_ERROR + +General authentication error. Use for issues like invalid API keys, insufficient permissions, or failed login attempts. + +#### AX\_VAL\_ERROR + +General validation error. Use when user-provided input fails validation checks (e.g., incorrect data type, missing required fields, invalid format. + +#### AX\_RES\_ERROR + +General resource error. Use for issues related to accessing or managing resources, such as a requested model being unavailable or quota limits exceeded. + +#### AX\_BIL\_ERROR + +General billing error. Use for problems related to billing, payments, or credits (e.g., insufficient funds, expired subscription. + +#### AX\_SUP\_ERROR + +General supplier error. Use when an error originates from an external supplier or third-party service integrated with aiXplain. + +#### AX\_NET\_ERROR + +General network error. Use for issues related to network connectivity, such as timeouts, DNS resolution failures, or unreachable services. + +#### AX\_SVC\_ERROR + +General service error. Use when a specific aiXplain service or endpoint is unavailable or malfunctioning (e.g., service downtime, internal component failure. + +#### AX\_INT\_ERROR + +General internal error. Use for unexpected server-side errors that are not covered by other categories. This often indicates a bug or an issue within the aiXplain platform itself. + +#### \_\_str\_\_ + +```python +def __str__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L88) + +Return the string representation of the error code. + +**Returns**: + +- `str` - The error code value as a string. + +### AixplainBaseException Objects + +```python +class AixplainBaseException(Exception) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L97) + +Base exception class for all aiXplain exceptions. + +This class serves as the foundation for all custom exceptions in the aiXplain +system. It provides structured error information including categorization, +severity, and additional context. + +**Attributes**: + +- `message` _str_ - Error message. +- `category` _ErrorCategory_ - Category of the error. +- `severity` _ErrorSeverity_ - Severity level of the error. +- `status_code` _Optional[int]_ - HTTP status code if applicable. +- `details` _Dict[str, Any]_ - Additional error context and details. +- `retry_recommended` _bool_ - Whether retrying the operation might succeed. +- `error_code` _Optional[ErrorCode]_ - Standardized error code. + +#### \_\_str\_\_ + +```python +def __str__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L133) + +Return a string representation of the exception. + +**Returns**: + +- `str` - Formatted string containing the exception class name, + error code (if present), and error message. + +#### to\_dict + +```python +def to_dict() -> Dict[str, Any] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L143) + +Convert the exception to a dictionary for serialization. + +**Returns**: + + Dict[str, Any]: Dictionary containing all exception attributes + including message, category, severity, status code, details, + retry recommendation, and error code. + +### AuthenticationError Objects + +```python +class AuthenticationError(AixplainBaseException) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L162) + +Raised when authentication fails. + +### ValidationError Objects + +```python +class ValidationError(AixplainBaseException) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L176) + +Raised when input validation fails. + +### ResourceError Objects + +```python +class ResourceError(AixplainBaseException) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L190) + +Raised when a resource is unavailable. + +### BillingError Objects + +```python +class BillingError(AixplainBaseException) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L204) + +Raised when there are billing issues. + +### SupplierError Objects + +```python +class SupplierError(AixplainBaseException) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L218) + +Raised when there are issues with external suppliers. + +### NetworkError Objects + +```python +class NetworkError(AixplainBaseException) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L232) + +Raised when there are network connectivity issues. + +### ServiceError Objects + +```python +class ServiceError(AixplainBaseException) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L246) + +Raised when a service is unavailable. + +### InternalError Objects + +```python +class InternalError(AixplainBaseException) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/exceptions/types.py#L260) + +Raised when there is an internal system error. + diff --git a/docs/api-reference/python/aixplain/factories/agent_factory/init.md b/docs/api-reference/python/aixplain/factories/agent_factory/init.md new file mode 100644 index 00000000..c13e186d --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/agent_factory/init.md @@ -0,0 +1,349 @@ +--- +sidebar_label: agent_factory +title: aixplain.factories.agent_factory +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Thiago Castro Ferreira and Lucas Pavanelli +Date: May 16th 2024 +Description: + Agent Factory Class + +### AgentFactory Objects + +```python +class AgentFactory() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/__init__.py#L51) + +Factory class for creating and managing agents in the aiXplain system. + +This class provides class methods for creating various types of agents and tools, +as well as managing existing agents in the platform. + +#### create + +```python +@classmethod +def create( + cls, + name: Text, + description: Text, + instructions: Optional[Text] = None, + llm: Optional[Union[LLM, Text]] = None, + llm_id: Optional[Text] = None, + tools: List[Union[Tool, Model]] = [], + api_key: Text = config.TEAM_API_KEY, + supplier: Union[Dict, Text, Supplier, int] = "aiXplain", + version: Optional[Text] = None, + tasks: List[AgentTask] = [], + output_format: Optional[OutputFormat] = None, + expected_output: Optional[Union[BaseModel, Text, + dict]] = None) -> Agent +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/__init__.py#L59) + +Create a new agent in the platform. + +**Warnings**: + + The 'instructions' parameter was recently added and serves the same purpose as 'description' did previously: set the role of the agent as a system prompt. + The 'description' parameter is still required and should be used to set a short summary of the agent's purpose. + For the next releases, the 'instructions' parameter will be required. + + +**Arguments**: + +- `name` _Text_ - name of the agent +- `description` _Text_ - description of the agent role. +- `instructions` _Text_ - instructions of the agent. +- `llm` _Optional[Union[LLM, Text]], optional_ - LLM instance to use as an object or as an ID. +- `llm_id` _Optional[Text], optional_ - ID of LLM to use if no LLM instance provided. Defaults to None. +- `tools` _List[Union[Tool, Model]], optional_ - list of tool for the agent. Defaults to []. +- `api_key` _Text, optional_ - team/user API key. Defaults to config.TEAM_API_KEY. +- `supplier` _Union[Dict, Text, Supplier, int], optional_ - owner of the agent. Defaults to "aiXplain". +- `version` _Optional[Text], optional_ - version of the agent. Defaults to None. +- `tasks` _List[AgentTask], optional_ - list of tasks for the agent. Defaults to []. +- `description`0 _OutputFormat, optional_ - default output format for agent responses. Defaults to OutputFormat.TEXT. +- `description`1 _Union[BaseModel, Text, dict], optional_ - expected output. Defaults to None. + +**Returns**: + +- `description`2 - created Agent + +#### create\_from\_dict + +```python +@classmethod +def create_from_dict(cls, dict: Dict) -> Agent +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/__init__.py#L191) + +Create an agent instance from a dictionary representation. + +**Arguments**: + +- `dict` _Dict_ - Dictionary containing agent configuration and properties. + + +**Returns**: + +- `Agent` - Instantiated agent object with properties from the dictionary. + + +**Raises**: + +- `Exception` - If agent validation fails or required properties are missing. + +#### create\_task + +```python +@classmethod +def create_task(cls, + name: Text, + description: Text, + expected_output: Text, + dependencies: Optional[List[Text]] = None) -> AgentTask +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/__init__.py#L209) + +Create a new task for an agent. + +**Arguments**: + +- `name` _Text_ - Name of the task. +- `description` _Text_ - Description of what the task should accomplish. +- `expected_output` _Text_ - Description of the expected output format. +- `dependencies` _Optional[List[Text]], optional_ - List of task names that must + complete before this task can start. Defaults to None. + + +**Returns**: + +- `AgentTask` - Created task object. + +#### create\_model\_tool + +```python +@classmethod +def create_model_tool(cls, + model: Optional[Union[Model, Text]] = None, + function: Optional[Union[Function, Text]] = None, + supplier: Optional[Union[Supplier, Text]] = None, + description: Text = "", + parameters: Optional[Dict] = None, + name: Optional[Text] = None) -> ModelTool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/__init__.py#L236) + +Create a new model tool for use with an agent. + +**Arguments**: + +- `model` _Optional[Union[Model, Text]], optional_ - Model instance or ID. Defaults to None. +- `function` _Optional[Union[Function, Text]], optional_ - Function enum or ID. Defaults to None. +- `supplier` _Optional[Union[Supplier, Text]], optional_ - Supplier enum or name. Defaults to None. +- `description` _Text, optional_ - Description of the tool. Defaults to "". +- `parameters` _Optional[Dict], optional_ - Tool parameters. Defaults to None. +- `name` _Optional[Text], optional_ - Name of the tool. Defaults to None. + + +**Returns**: + +- `ModelTool` - Created model tool object. + + +**Raises**: + +- `AssertionError` - If the supplier is not valid. + +#### create\_pipeline\_tool + +```python +@classmethod +def create_pipeline_tool(cls, + description: Text, + pipeline: Union[Pipeline, Text], + name: Optional[Text] = None) -> PipelineTool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/__init__.py#L284) + +Create a new pipeline tool for use with an agent. + +**Arguments**: + +- `description` _Text_ - Description of what the pipeline tool does. +- `pipeline` _Union[Pipeline, Text]_ - Pipeline instance or pipeline ID. +- `name` _Optional[Text], optional_ - Name of the tool. Defaults to None. + + +**Returns**: + +- `PipelineTool` - Created pipeline tool object. + +#### create\_python\_interpreter\_tool + +```python +@classmethod +def create_python_interpreter_tool(cls) -> PythonInterpreterTool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/__init__.py#L303) + +Create a new Python interpreter tool for use with an agent. + +This tool allows the agent to execute Python code in a controlled environment. + +**Returns**: + +- `PythonInterpreterTool` - Created Python interpreter tool object. + +#### create\_custom\_python\_code\_tool + +```python +@classmethod +def create_custom_python_code_tool( + cls, + code: Union[Text, Callable], + name: Text, + description: Text = "") -> CustomPythonCodeTool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/__init__.py#L314) + +Create a new custom Python code tool for use with an agent. + +**Arguments**: + +- `code` _Union[Text, Callable]_ - Python code as string or callable function. +- `name` _Text_ - Name of the tool. +- `description` _Text, optional_ - Description of what the tool does. Defaults to "". + + +**Returns**: + +- `CustomPythonCodeTool` - Created custom Python code tool object. + +#### create\_sql\_tool + +```python +@classmethod +def create_sql_tool(cls, + name: Text, + description: Text, + source: str, + source_type: Union[str, DatabaseSourceType], + schema: Optional[Text] = None, + tables: Optional[List[Text]] = None, + enable_commit: bool = False) -> SQLTool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/__init__.py#L330) + +Create a new SQL tool + +**Arguments**: + +- `name` _Text_ - name of the tool +- `description` _Text_ - description of the database tool +- `source` _Union[Text, Dict]_ - database source - can be a connection string or dictionary with connection details +- `source_type` _Union[str, DatabaseSourceType]_ - type of source (postgresql, sqlite, csv) or DatabaseSourceType enum +- `schema` _Optional[Text], optional_ - database schema description +- `tables` _Optional[List[Text]], optional_ - table names to work with (optional) +- `enable_commit` _bool, optional_ - enable to modify the database (optional) + +**Returns**: + +- `SQLTool` - created SQLTool + + +**Examples**: + + # CSV - Simple + sql_tool = AgentFactory.create_sql_tool( + description="My CSV Tool", + source="/path/to/data.csv", + source_type="csv", + tables=["data"] + ) + + # SQLite - Simple + sql_tool = AgentFactory.create_sql_tool( + description="My SQLite Tool", + source="/path/to/database.sqlite", + source_type="sqlite", + tables=["users", "products"] + ) + +#### list + +```python +@classmethod +def list(cls) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/__init__.py#L463) + +List all agents available in the platform. + +**Returns**: + +- `Dict` - Dictionary containing: + - results (List[Agent]): List of available agents. + - page_total (int): Number of agents in current page. + - page_number (int): Current page number. + - total (int): Total number of agents. + + +**Raises**: + +- `Exception` - If there is an error listing the agents. + +#### get + +```python +@classmethod +def get(cls, agent_id: Text, api_key: Optional[Text] = None) -> Agent +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/__init__.py#L513) + +Retrieve an agent by its ID. + +**Arguments**: + +- `agent_id` _Text_ - ID of the agent to retrieve. +- `api_key` _Optional[Text], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + +- `Agent` - Retrieved agent object. + + +**Raises**: + +- `Exception` - If the agent cannot be retrieved or doesn't exist. + diff --git a/docs/api-reference/python/aixplain/factories/agent_factory/utils.md b/docs/api-reference/python/aixplain/factories/agent_factory/utils.md new file mode 100644 index 00000000..eb9e20e1 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/agent_factory/utils.md @@ -0,0 +1,101 @@ +--- +sidebar_label: utils +title: aixplain.factories.agent_factory.utils +--- + +#### build\_tool\_payload + +```python +def build_tool_payload(tool: Union[Tool, Model]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/utils.py#L26) + +Build a tool payload from a tool or model object. + +**Arguments**: + +- `tool` _Union[Tool, Model]_ - The tool or model object to build the payload from. + + +**Returns**: + +- `Dict` - The tool payload. + +#### build\_tool + +```python +def build_tool(tool: Dict) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/utils.py#L56) + +Build a tool from a dictionary. + +**Arguments**: + +- `tool` _Dict_ - Tool dictionary. + + +**Returns**: + +- `Tool` - Tool object. + +#### build\_llm + +```python +def build_llm(payload: Dict, api_key: Text = config.TEAM_API_KEY) -> LLM +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/utils.py#L119) + +Build a Large Language Model (LLM) instance from a dictionary configuration. + +This function attempts to create an LLM instance either from a cached LLM object +in the payload or by creating a new instance using the provided configuration. + +**Arguments**: + +- `payload` _Dict_ - Dictionary containing LLM configuration and possibly a cached + LLM object. +- `api_key` _Text, optional_ - API key for authentication. Defaults to config.TEAM_API_KEY. + + +**Returns**: + +- `LLM` - Instantiated LLM object with configured parameters. + +#### build\_agent + +```python +def build_agent(payload: Dict, + tools: List[Tool] = None, + api_key: Text = config.TEAM_API_KEY) -> Agent +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/agent_factory/utils.py#L170) + +Build an agent instance from a dictionary configuration. + +This function creates an agent with its associated tools, LLM, and tasks based +on the provided configuration. + +**Arguments**: + +- `payload` _Dict_ - Dictionary containing agent configuration including tools, + LLM settings, and tasks. +- `tools` _List[Tool], optional_ - List of pre-configured tools to use. If None, + tools will be built from the payload. Defaults to None. +- `api_key` _Text, optional_ - API key for authentication. Defaults to config.TEAM_API_KEY. + + +**Returns**: + +- `Agent` - Instantiated agent object with configured tools, LLM, and tasks. + + +**Raises**: + +- `ValueError` - If a tool type is not supported. +- `AssertionError` - If tool configuration is invalid. + diff --git a/docs/api-reference/python/aixplain/factories/api_key_factory.md b/docs/api-reference/python/aixplain/factories/api_key_factory.md new file mode 100644 index 00000000..e1beb46e --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/api_key_factory.md @@ -0,0 +1,187 @@ +--- +sidebar_label: api_key_factory +title: aixplain.factories.api_key_factory +--- + +### APIKeyFactory Objects + +```python +class APIKeyFactory() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/api_key_factory.py#L10) + +Factory class for managing API keys in the aiXplain platform. + +This class provides functionality for creating, retrieving, updating, and +monitoring API keys, including their usage limits and budgets. + +**Attributes**: + +- `backend_url` _str_ - Base URL for the aiXplain backend API. + +#### get + +```python +@classmethod +def get(cls, api_key: Text) -> APIKey +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/api_key_factory.py#L22) + +Retrieve an API key by its value. + +This method searches for an API key by matching the first and last 4 +characters of the provided key. + +**Arguments**: + +- `api_key` _Text_ - The API key value to search for. + + +**Returns**: + +- `APIKey` - The matching API key object. + + +**Raises**: + +- `Exception` - If no matching API key is found. + +#### list + +```python +@classmethod +def list(cls) -> List[APIKey] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/api_key_factory.py#L43) + +List all API keys accessible to the current user. + +This method retrieves all API keys that the authenticated user has access to, +using the configured TEAM_API_KEY. + +**Returns**: + +- `List[APIKey]` - List of API key objects. + + +**Raises**: + +- `Exception` - If the API request fails or returns an error, including cases + where authentication fails or the service is unavailable. + +#### create + +```python +@classmethod +def create(cls, name: Text, budget: int, global_limits: Union[Dict, + APIKeyLimits], + asset_limits: List[Union[Dict, APIKeyLimits]], + expires_at: datetime) -> APIKey +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/api_key_factory.py#L85) + +Create a new API key with specified limits and budget. + +This method creates a new API key with configured usage limits, budget, +and expiration date. + +**Arguments**: + +- `name` _Text_ - Name or description for the API key. +- `budget` _int_ - Total budget allocated to this API key. +- `global_limits` _Union[Dict, APIKeyLimits]_ - Global usage limits for the key, + either as a dictionary or APIKeyLimits object. +- `asset_limits` _List[Union[Dict, APIKeyLimits]]_ - List of per-asset usage + limits, each either as a dictionary or APIKeyLimits object. +- `expires_at` _datetime_ - Expiration date and time for the API key. + + +**Returns**: + +- `APIKey` - Created API key object with its access key and configuration. + + +**Raises**: + +- `Exception` - If the API request fails or returns an error, including cases + where validation fails or the service is unavailable. + +#### update + +```python +@classmethod +def update(cls, api_key: APIKey) -> APIKey +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/api_key_factory.py#L145) + +Update an existing API key's configuration. + +This method updates an API key's settings such as limits, budget, and +expiration date. The API key must be validated before update. + +**Arguments**: + +- `api_key` _APIKey_ - API key object with updated configuration. + Must have a valid ID of an existing key. + + +**Returns**: + +- `APIKey` - Updated API key object with new configuration. + + +**Raises**: + +- `Exception` - If: + - API key validation fails + - API key ID is invalid + - Update request fails + - Service is unavailable + +#### get\_usage\_limits + +```python +@classmethod +def get_usage_limits( + cls, + api_key: Text = config.TEAM_API_KEY, + asset_id: Optional[Text] = None) -> List[APIKeyUsageLimit] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/api_key_factory.py#L194) + +Retrieve current usage limits and counts for an API key. + +This method fetches the current usage statistics and limits for an API key, +optionally filtered by a specific asset. + +**Arguments**: + +- `api_key` _Text, optional_ - API key to check usage for. Defaults to + config.TEAM_API_KEY. +- `asset_id` _Optional[Text], optional_ - Filter usage limits for a specific + asset. Defaults to None, showing all assets. + + +**Returns**: + +- `List[APIKeyUsageLimit]` - List of usage limit objects containing: + - daily_request_count: Current number of requests today + - daily_request_limit: Maximum allowed requests per day + - daily_token_count: Current number of tokens used today + - daily_token_limit: Maximum allowed tokens per day + - model: Asset ID if limit is asset-specific, None if global + + +**Raises**: + +- `Exception` - If: + - API key is invalid + - User is not the key owner + - Service is unavailable + diff --git a/docs/api-reference/python/aixplain/factories/asset_factory.md b/docs/api-reference/python/aixplain/factories/asset_factory.md new file mode 100644 index 00000000..1f556d3e --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/asset_factory.md @@ -0,0 +1,64 @@ +--- +sidebar_label: asset_factory +title: aixplain.factories.asset_factory +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: December 27th 2022 +Description: + Asset Factory Class + +### AssetFactory Objects + +```python +class AssetFactory() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/asset_factory.py#L30) + +Base class for asset factories. + +This class provides a common interface for creating and retrieving assets +from the aiXplain platform. Subclasses should implement the abstract methods +to define specific asset types. + +**Attributes**: + +- `backend_url` _str_ - Base URL for the aiXplain backend API. + +#### get + +```python +@abstractmethod +def get(asset_id: Text) -> Asset +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/asset_factory.py#L43) + +Create a 'Asset' object from id + +**Arguments**: + +- `asset_id` _str_ - ID of required asset. + + +**Returns**: + +- `Asset` - Created 'Asset' object + diff --git a/docs/api-reference/python/aixplain/factories/benchmark_factory.md b/docs/api-reference/python/aixplain/factories/benchmark_factory.md new file mode 100644 index 00000000..604c4cf3 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/benchmark_factory.md @@ -0,0 +1,204 @@ +--- +sidebar_label: benchmark_factory +title: aixplain.factories.benchmark_factory +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: December 2nd 2022 +Description: + Benchmark Factory Class + +### BenchmarkFactory Objects + +```python +class BenchmarkFactory() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/benchmark_factory.py#L39) + +Factory class for creating and managing benchmarks in the aiXplain platform. + +This class provides functionality for creating benchmarks, managing benchmark jobs, +retrieving results, and configuring normalization options. Benchmarks can be used +to evaluate and compare multiple models using specified datasets and metrics. + +**Attributes**: + +- `backend_url` _str_ - Base URL for the aiXplain backend API. + +#### get + +```python +@classmethod +def get(cls, benchmark_id: str) -> Benchmark +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/benchmark_factory.py#L121) + +Retrieve a benchmark by its ID. + +This method fetches a benchmark and all its associated components +(models, datasets, metrics, jobs) from the platform. + +**Arguments**: + +- `benchmark_id` _str_ - Unique identifier of the benchmark to retrieve. + + +**Returns**: + +- `Benchmark` - Retrieved benchmark object with all components loaded. + + +**Raises**: + +- `Exception` - If: + - Benchmark ID is invalid + - Authentication fails + - Service is unavailable + +#### get\_job + +```python +@classmethod +def get_job(cls, job_id: Text) -> BenchmarkJob +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/benchmark_factory.py#L170) + +Retrieve a benchmark job by its ID. + +**Arguments**: + +- `job_id` _Text_ - Unique identifier of the benchmark job to retrieve. + + +**Returns**: + +- `BenchmarkJob` - Retrieved benchmark job object with its current status. + + +**Raises**: + +- `Exception` - If the job ID is invalid or the request fails. + +#### create + +```python +@classmethod +def create(cls, name: str, dataset_list: List[Dataset], + model_list: List[Model], metric_list: List[Metric]) -> Benchmark +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/benchmark_factory.py#L254) + +Create a new benchmark configuration. + +This method creates a new benchmark that can be used to evaluate and compare +multiple models using specified datasets and metrics. Note that this only +creates the benchmark configuration - you need to run it separately using +start_benchmark_job. + +**Arguments**: + +- `name` _str_ - Unique name for the benchmark. +- `dataset_list` _List[Dataset]_ - List of datasets to use for evaluation. + Currently only supports a single dataset. +- `model_list` _List[Model]_ - List of models to evaluate. All models must + either have additional configuration info or none should have it. +- `metric_list` _List[Metric]_ - List of metrics to use for evaluation. + Must provide at least one metric. + + +**Returns**: + +- `Benchmark` - Created benchmark object ready for execution. + + +**Raises**: + +- `Exception` - If: + - No dataset is provided or multiple datasets are provided + - No metrics are provided + - No models are provided + - Model configuration is inconsistent + - Request fails or returns an error + +#### list\_normalization\_options + +```python +@classmethod +def list_normalization_options(cls, metric: Metric, model: Model) -> List[str] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/benchmark_factory.py#L326) + +List supported normalization options for a metric-model pair. + +This method retrieves the list of normalization options that can be used +when evaluating a specific model with a specific metric in a benchmark. + +**Arguments**: + +- `metric` _Metric_ - Metric to get normalization options for. +- `model` _Model_ - Model to check compatibility with. + + +**Returns**: + +- `List[str]` - List of supported normalization option identifiers. + + +**Raises**: + +- `Exception` - If: + - Metric or model is invalid + - Request fails + - Service is unavailable + +#### get\_benchmark\_job\_scores + +```python +@classmethod +def get_benchmark_job_scores(cls, job_id: Text) -> Any +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/benchmark_factory.py#L370) + +Retrieve and format benchmark job scores. + +This method fetches the scores from a benchmark job and formats them into +a pandas DataFrame, with model names properly formatted to include supplier +and version information. + +**Arguments**: + +- `job_id` _Text_ - Unique identifier of the benchmark job. + + +**Returns**: + +- `pandas.DataFrame` - DataFrame containing benchmark scores with formatted + model names. + + +**Raises**: + +- `Exception` - If the job ID is invalid or the request fails. + diff --git a/docs/api-reference/python/aixplain/factories/cli/model_factory_cli.md b/docs/api-reference/python/aixplain/factories/cli/model_factory_cli.md new file mode 100644 index 00000000..92405b9f --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/cli/model_factory_cli.md @@ -0,0 +1,317 @@ +--- +sidebar_label: model_factory_cli +title: aixplain.factories.cli.model_factory_cli +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Michael Lam +Date: September 18th 2023 +Description: + Model Factory CLI + +#### list\_host\_machines + +```python +@click.command("hosts") +@click.option("--api-key", + default=None, + help="TEAM_API_KEY if not already set in environment") +def list_host_machines(api_key: Optional[Text] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/cli/model_factory_cli.py#L32) + +List available host machines for model deployment. + +This CLI command wraps the ModelFactory.list_host_machines function and outputs +the results in YAML format. + +**Arguments**: + +- `api_key` _Text, optional_ - Team API key for authentication. Defaults to None, + using the configured environment variable. + + +**Returns**: + +- `None` - Prints the host machines list in YAML format to stdout. + +#### list\_functions + +```python +@click.command("functions") +@click.option("--verbose", + is_flag=True, + help="List all function details, False by default.") +@click.option("--api-key", + default=None, + help="TEAM_API_KEY if not already set in environment.") +def list_functions(verbose: bool, api_key: Optional[Text] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/cli/model_factory_cli.py#L53) + +List available functions for model deployment. + +This CLI command wraps the ModelFactory.list_functions function and outputs +the results in YAML format. Functions represent the different types of +operations that models can perform. + +**Arguments**: + +- `verbose` _bool_ - If True, includes detailed information about each function. + If False, provides a simplified list. +- `api_key` _Text, optional_ - Team API key for authentication. Defaults to None, + using the configured environment variable. + + +**Returns**: + +- `None` - Prints the functions list in YAML format to stdout. + +#### list\_gpus + +```python +@click.command("gpus") +@click.option("--api-key", + default=None, + help="TEAM_API_KEY if not already set in environment.") +def list_gpus(api_key: Optional[Text] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/cli/model_factory_cli.py#L76) + +List available GPUs for model deployment. + +This CLI command wraps the ModelFactory.list_gpus function and outputs +the results in YAML format. Shows available GPU resources that can be +used for model hosting. + +**Arguments**: + +- `api_key` _Text, optional_ - Team API key for authentication. Defaults to None, + using the configured environment variable. + + +**Returns**: + +- `None` - Prints the GPU list in YAML format to stdout. + +#### create\_asset\_repo + +```python +@click.command("image-repo") +@click.option("--name", help="Model name.") +@click.option("--description", help="Description of model.") +@click.option("--function", help="Function name obtained from LIST_FUNCTIONS.") +@click.option( + "--source-language", + default="en", + help= + "Model source language in 2-character 639-1 code or 3-character 639-3 code." +) +@click.option("--input-modality", help="Input type (text, video, image, etc.)") +@click.option("--output-modality", + help="Output type (text, video, image, etc.)") +@click.option("--documentation-url", + default="", + help="Link to model documentation.") +@click.option("--api-key", + default=None, + help="TEAM_API_KEY if not already set in environment.") +def create_asset_repo(name: Text, + description: Text, + function: Text, + source_language: Text, + input_modality: Text, + output_modality: Text, + documentation_url: Optional[Text] = "", + api_key: Optional[Text] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/cli/model_factory_cli.py#L106) + +Create a new asset repository for a model. + +This CLI command wraps the ModelFactory.create_asset_repo function and outputs +the results in YAML format. Creates a new repository for storing model assets +and configurations. + +**Arguments**: + +- `name` _Text_ - Name of the model. +- `description` _Text_ - Description of the model's purpose and functionality. +- `function` _Text_ - Model function name obtained via list_functions. +- `source_language` _Text_ - Language code in ISO 639-1 (2-char) or 639-3 (3-char) format. +- `input_modality` _Text_ - Type of input the model accepts (e.g., text, video, image). +- `output_modality` _Text_ - Type of output the model produces (e.g., text, video, image). +- `documentation_url` _Text, optional_ - URL to model documentation. Defaults to "". +- `api_key` _Text, optional_ - Team API key for authentication. Defaults to None, + using the configured environment variable. + + +**Returns**: + +- `None` - Prints the created repository details in YAML format to stdout. + +#### asset\_repo\_login + +```python +@click.command("image-repo-login") +@click.option("--api-key", + default=None, + help="TEAM_API_KEY if not already set in environment.") +def asset_repo_login(api_key: Optional[Text] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/cli/model_factory_cli.py#L145) + +Get login credentials for the asset repository. + +This CLI command wraps the ModelFactory.asset_repo_login function and outputs +the results in YAML format. Provides authentication details needed to access +the model asset repository. + +**Arguments**: + +- `api_key` _Text, optional_ - Team API key for authentication. Defaults to None, + using the configured environment variable. + + +**Returns**: + +- `None` - Prints the login credentials in YAML format to stdout. + +#### onboard\_model + +```python +@click.command("model") +@click.option("--model-id", help="Model ID from CREATE_IMAGE_REPO.") +@click.option("--image-tag", + help="The tag of the image that you would like hosted.") +@click.option("--image-hash", + help="The hash of the image you would like onboarded.") +@click.option("--host-machine", + default="", + help="The machine on which to host the model.") +@click.option("--api-key", + default=None, + help="TEAM_API_KEY if not already set in environment.") +def onboard_model(model_id: Text, + image_tag: Text, + image_hash: Text, + host_machine: Optional[Text] = "", + api_key: Optional[Text] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/cli/model_factory_cli.py#L170) + +Onboard a model image for deployment. + +This CLI command wraps the ModelFactory.onboard_model function and outputs +the results in YAML format. Prepares a model image for deployment by registering +it with the platform. + +**Arguments**: + +- `model_id` _Text_ - Model ID obtained from create_asset_repo. +- `image_tag` _Text_ - Tag of the Docker image to be onboarded. +- `image_hash` _Text_ - Hash of the Docker image for verification. +- `host_machine` _Text, optional_ - ID of the machine to host the model. Defaults to "". +- `api_key` _Text, optional_ - Team API key for authentication. Defaults to None, + using the configured environment variable. + + +**Returns**: + +- `None` - Prints the onboarding results in YAML format to stdout. + +#### deploy\_huggingface\_model + +```python +@click.command("hf-model") +@click.option("--name", help="User-defined name for Hugging Face model.") +@click.option( + "--hf-repo-id", + help="Repository ID from Hugging Face in {supplier}/{model name} form.") +@click.option("--revision", default="", help="Commit hash of repository.") +@click.option("--hf-token", + default=None, + help="Hugging Face token used to authenticate to this model.") +@click.option("--api-key", + default=None, + help="TEAM_API_KEY if not already set in environment.") +def deploy_huggingface_model(name: Text, + hf_repo_id: Text, + hf_token: Optional[Text] = None, + revision: Optional[Text] = None, + api_key: Optional[Text] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/cli/model_factory_cli.py#L201) + +Deploy a model from Hugging Face Hub. + +This CLI command wraps the ModelFactory.deploy_huggingface_model function and outputs +the results in YAML format. Deploys a model directly from Hugging Face's model hub. + +**Arguments**: + +- `name` _Text_ - User-defined name for the Hugging Face model. +- `hf_repo_id` _Text_ - Repository ID from Hugging Face in 'org/model-name' format. +- `hf_token` _Text, optional_ - Hugging Face token for private models. Defaults to None. +- `revision` _Text, optional_ - Specific model revision/commit hash. Defaults to None. +- `api_key` _Text, optional_ - Team API key for authentication. Defaults to None, + using the configured environment variable. + + +**Returns**: + +- `None` - Prints the deployment results in YAML format to stdout. + +#### get\_huggingface\_model\_status + +```python +@click.command("hf-model-status") +@click.option("--model-id", help="Model ID from DEPLOY_HUGGINGFACE_MODEL.") +@click.option("--api-key", + default=None, + help="TEAM_API_KEY if not already set in environment.") +def get_huggingface_model_status(model_id: Text, + api_key: Optional[Text] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/cli/model_factory_cli.py#L232) + +Check the deployment status of a Hugging Face model. + +This CLI command wraps the ModelFactory.get_huggingface_model_status function and +outputs the results in YAML format. Retrieves the current status of a Hugging Face +model deployment. + +**Arguments**: + +- `model_id` _Text_ - Model ID obtained from deploy_huggingface_model. +- `api_key` _Text, optional_ - Team API key for authentication. Defaults to None, + using the configured environment variable. + + +**Returns**: + +- `None` - Prints the model status in YAML format to stdout. + diff --git a/docs/api-reference/python/aixplain/factories/corpus_factory.md b/docs/api-reference/python/aixplain/factories/corpus_factory.md new file mode 100644 index 00000000..6cf0d944 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/corpus_factory.md @@ -0,0 +1,231 @@ +--- +sidebar_label: corpus_factory +title: aixplain.factories.corpus_factory +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: March 27th 2023 +Description: + Corpus Factory Class + +### CorpusFactory Objects + +```python +class CorpusFactory(AssetFactory) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/corpus_factory.py#L50) + +Factory class for creating and managing corpora in the aiXplain platform. + +This class provides functionality for creating, retrieving, and managing +corpora, which are collections of data assets used for training and +evaluating AI models. + +**Attributes**: + +- `backend_url` _str_ - Base URL for the aiXplain backend API. + +#### get + +```python +@classmethod +def get(cls, corpus_id: Text) -> Corpus +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/corpus_factory.py#L128) + +Retrieve a corpus by its ID. + +This method fetches a corpus and all its associated data assets from +the platform. + +**Arguments**: + +- `corpus_id` _Text_ - Unique identifier of the corpus to retrieve. + + +**Returns**: + +- `Corpus` - Retrieved corpus object with all data assets loaded. + + +**Raises**: + +- `Exception` - If: + - Corpus ID is invalid + - Authentication fails + - Service is unavailable + +#### list + +```python +@classmethod +def list(cls, + query: Optional[Text] = None, + function: Optional[Function] = None, + language: Optional[Union[Language, List[Language]]] = None, + data_type: Optional[DataType] = None, + license: Optional[License] = None, + page_number: int = 0, + page_size: int = 20) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/corpus_factory.py#L176) + +List and filter corpora with pagination support. + +This method provides comprehensive filtering and pagination capabilities +for retrieving corpora from the aiXplain platform. + +**Arguments**: + +- `query` _Optional[Text], optional_ - Search query to filter corpora by name + or description. Defaults to None. +- `function` _Optional[Function], optional_ - Filter by AI function type. + Defaults to None. +- `language` _Optional[Union[Language, List[Language]]], optional_ - Filter by + language(s). Can be single language or list. Defaults to None. +- `data_type` _Optional[DataType], optional_ - Filter by data type. + Defaults to None. +- `license` _Optional[License], optional_ - Filter by license type. + Defaults to None. +- `page_number` _int, optional_ - Zero-based page number. Defaults to 0. +- `page_size` _int, optional_ - Number of items per page (1-100). + Defaults to 20. + + +**Returns**: + +- `Dict` - Response containing: + - results (List[Corpus]): List of corpus objects + - page_total (int): Total items in current page + - page_number (int): Current page number + - total (int): Total number of items across all pages + + +**Raises**: + +- `Exception` - If: + - page_size is not between 1 and 100 + - Request fails + - Service is unavailable +- `AssertionError` - If page_size is invalid. + +#### get\_assets\_from\_page + +```python +@classmethod +def get_assets_from_page(cls, + page_number: int = 1, + task: Optional[Function] = None, + language: Optional[Text] = None) -> List[Corpus] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/corpus_factory.py#L278) + +Retrieve a paginated list of corpora with optional filters. + +**Notes**: + + This method is deprecated. Use list() instead. + + +**Arguments**: + +- `page_number` _int, optional_ - One-based page number. Defaults to 1. +- `task` _Optional[Function], optional_ - Filter by AI task/function. + Defaults to None. +- `language` _Optional[Text], optional_ - Filter by language code. + Defaults to None. + + +**Returns**: + +- `List[Corpus]` - List of corpus objects matching the filters. + + Deprecated: + Use list() method instead for more comprehensive filtering and + pagination capabilities. + +#### create + +```python +@classmethod +def create(cls, + name: Text, + description: Text, + license: License, + content_path: Union[Union[Text, Path], List[Union[Text, Path]]], + schema: List[Union[Dict, MetaData]], + ref_data: List[Any] = [], + tags: List[Text] = [], + functions: List[Function] = [], + privacy: Privacy = Privacy.PRIVATE, + error_handler: ErrorHandler = ErrorHandler.SKIP, + api_key: Optional[Text] = None) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/corpus_factory.py#L310) + +Create a new corpus from data files. + +This method asynchronously uploads and processes data files to create a new +corpus in the user's dashboard. The data files are processed according to +the provided schema and combined with any referenced existing data. + +**Arguments**: + +- `name` _Text_ - Name for the new corpus. +- `description` _Text_ - Description of the corpus's contents and purpose. +- `license` _License_ - License type for the corpus. +- `content_path` _Union[Union[Text, Path], List[Union[Text, Path]]]_ - Path(s) + to CSV files containing the data. Can be single path or list. +- `schema` _List[Union[Dict, MetaData]]_ - Metadata configurations defining + how to process the data files. +- `ref_data` _List[Any], optional_ - References to existing data assets to + include in the corpus. Can be Data objects or IDs. Defaults to []. +- `tags` _List[Text], optional_ - Tags describing the corpus content. + Defaults to []. +- `functions` _List[Function], optional_ - AI functions this corpus is + suitable for. Defaults to []. +- `privacy` _Privacy, optional_ - Visibility setting for the corpus. + Defaults to Privacy.PRIVATE. +- `error_handler` _ErrorHandler, optional_ - Strategy for handling data + processing errors. Defaults to ErrorHandler.SKIP. +- `description`0 _Optional[Text], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + +- `description`1 - Response containing: + - status: Current processing status + - asset_id: ID of the created corpus + + +**Raises**: + +- `description`2 - If: + - No schema or reference data provided + - Referenced data asset doesn't exist + - Reserved column names are used + - Data rows are misaligned + - Processing or upload fails + diff --git a/docs/api-reference/python/aixplain/factories/data_factory.md b/docs/api-reference/python/aixplain/factories/data_factory.md new file mode 100644 index 00000000..d79daa76 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/data_factory.md @@ -0,0 +1,76 @@ +--- +sidebar_label: data_factory +title: aixplain.factories.data_factory +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: May 15th 2023 +Description: + Data Factory Class + +### DataFactory Objects + +```python +class DataFactory(AssetFactory) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/data_factory.py#L38) + +Factory class for creating and managing data assets. + +This class provides functionality for creating, retrieving, and managing +data assets in the aiXplain platform. Data assets represent individual +pieces of data (e.g., text, audio) that can be used in corpora or +directly with models. + +**Attributes**: + +- `backend_url` _str_ - Base URL for the aiXplain backend API. + +#### get + +```python +@classmethod +def get(cls, data_id: Text) -> Data +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/data_factory.py#L95) + +Retrieve a data asset by its ID. + +This method fetches a data asset from the platform using its unique +identifier. + +**Arguments**: + +- `data_id` _Text_ - Unique identifier of the data asset to retrieve. + + +**Returns**: + +- `Data` - Retrieved data asset object with its configuration. + + +**Raises**: + +- `Exception` - If: + - Data asset ID is invalid or not found + - Authentication fails + - Service is unavailable + diff --git a/docs/api-reference/python/aixplain/factories/dataset_factory.md b/docs/api-reference/python/aixplain/factories/dataset_factory.md new file mode 100644 index 00000000..0455b21c --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/dataset_factory.md @@ -0,0 +1,239 @@ +--- +sidebar_label: dataset_factory +title: aixplain.factories.dataset_factory +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: December 1st 2022 +Description: + Dataset Factory Class + +### DatasetFactory Objects + +```python +class DatasetFactory(AssetFactory) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/dataset_factory.py#L54) + +Factory class for creating and managing datasets in the aiXplain platform. + +This class provides functionality for creating, retrieving, and managing +datasets, which are structured collections of data assets used for training, +evaluating, and benchmarking AI models. Datasets can include input data, +target data, hypotheses, and metadata. + +**Attributes**: + +- `backend_url` _str_ - Base URL for the aiXplain backend API. + +#### get + +```python +@classmethod +def get(cls, dataset_id: Text) -> Dataset +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/dataset_factory.py#L177) + +Retrieve a dataset by its ID. + +This method fetches a dataset and all its associated data assets from +the platform. + +**Arguments**: + +- `dataset_id` _Text_ - Unique identifier of the dataset to retrieve. + + +**Returns**: + +- `Dataset` - Retrieved dataset object with all components loaded. + + +**Raises**: + +- `Exception` - If: + - Dataset ID is invalid + - Authentication fails + - Service is unavailable + +#### list + +```python +@classmethod +def list(cls, + query: Optional[Text] = None, + function: Optional[Function] = None, + source_languages: Optional[Union[Language, List[Language]]] = None, + target_languages: Optional[Union[Language, List[Language]]] = None, + data_type: Optional[DataType] = None, + license: Optional[License] = None, + is_referenceless: Optional[bool] = None, + page_number: int = 0, + page_size: int = 20) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/dataset_factory.py#L216) + +List and filter datasets with pagination support. + +This method provides comprehensive filtering and pagination capabilities +for retrieving datasets from the aiXplain platform. + +**Arguments**: + +- `query` _Optional[Text], optional_ - Search query to filter datasets by name + or description. Defaults to None. +- `function` _Optional[Function], optional_ - Filter by AI function type. + Defaults to None. + source_languages (Optional[Union[Language, List[Language]]], optional): + Filter by input data language(s). Can be single language or list. + Defaults to None. + target_languages (Optional[Union[Language, List[Language]]], optional): + Filter by output data language(s). Can be single language or list. + Defaults to None. +- `data_type` _Optional[DataType], optional_ - Filter by data type. + Defaults to None. +- `license` _Optional[License], optional_ - Filter by license type. + Defaults to None. +- `is_referenceless` _Optional[bool], optional_ - Filter by whether dataset + has references. Defaults to None. +- `page_number` _int, optional_ - Zero-based page number. Defaults to 0. +- `page_size` _int, optional_ - Number of items per page (1-100). + Defaults to 20. + + +**Returns**: + +- `Dict` - Response containing: + - results (List[Dataset]): List of dataset objects + - page_total (int): Total items in current page + - page_number (int): Current page number + - total (int): Total number of items across all pages + + +**Raises**: + +- `Exception` - If: + - page_size is not between 1 and 100 + - Request fails + - Service is unavailable +- `AssertionError` - If page_size is invalid. + +#### create + +```python +@classmethod +def create(cls, + name: Text, + description: Text, + license: License, + function: Function, + input_schema: List[Union[Dict, MetaData]], + output_schema: List[Union[Dict, MetaData]] = [], + hypotheses_schema: List[Union[Dict, MetaData]] = [], + metadata_schema: List[Union[Dict, MetaData]] = [], + content_path: Union[Union[Text, Path], List[Union[Text, + Path]]] = [], + input_ref_data: Dict[Text, Any] = {}, + output_ref_data: Dict[Text, List[Any]] = {}, + hypotheses_ref_data: Dict[Text, Any] = {}, + meta_ref_data: Dict[Text, Any] = {}, + tags: List[Text] = [], + privacy: Privacy = Privacy.PRIVATE, + split_labels: Optional[List[Text]] = None, + split_rate: Optional[List[float]] = None, + error_handler: ErrorHandler = ErrorHandler.SKIP, + s3_link: Optional[Text] = None, + aws_credentials: Optional[Dict[Text, Text]] = { + "AWS_ACCESS_KEY_ID": None, + "AWS_SECRET_ACCESS_KEY": None + }, + api_key: Optional[Text] = None) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/dataset_factory.py#L331) + +Create a new dataset from data files and references. + +This method processes data files and existing data assets to create a new +dataset in the platform. It supports various data types, multiple input and +output configurations, and optional data splitting. + +**Arguments**: + +- `name` _Text_ - Name for the new dataset. +- `description` _Text_ - Description of the dataset's contents and purpose. +- `license` _License_ - License type for the dataset. +- `function` _Function_ - AI function this dataset is suitable for. +- `input_schema` _List[Union[Dict, MetaData]]_ - Metadata configurations for + input data processing. +- `output_schema` _List[Union[Dict, MetaData]], optional_ - Metadata configs + for output/target data. Defaults to []. +- `hypotheses_schema` _List[Union[Dict, MetaData]], optional_ - Metadata + configs for hypothesis data. Defaults to []. +- `metadata_schema` _List[Union[Dict, MetaData]], optional_ - Additional + metadata configurations. Defaults to []. + content_path (Union[Union[Text, Path], List[Union[Text, Path]]], optional): + Path(s) to data files. Can be single path or list. Defaults to []. +- `input_ref_data` _Dict[Text, Any], optional_ - References to existing + input data assets. Defaults to \{}. +- `output_ref_data` _Dict[Text, List[Any]], optional_ - References to + existing output data assets. Defaults to \{}. +- `description`0 _Dict[Text, Any], optional_ - References to + existing hypothesis data. Defaults to \{}. +- `description`1 _Dict[Text, Any], optional_ - References to existing + metadata assets. Defaults to \{}. +- `description`2 _List[Text], optional_ - Tags describing the dataset. + Defaults to []. +- `description`3 _Privacy, optional_ - Visibility setting. + Defaults to Privacy.PRIVATE. +- `description`4 _Optional[List[Text]], optional_ - Labels for dataset + splits (e.g., ["train", "test"]). Defaults to None. +- `description`5 _Optional[List[float]], optional_ - Ratios for dataset + splits (must sum to 1). Defaults to None. +- `description`6 _ErrorHandler, optional_ - Strategy for handling data + processing errors. Defaults to ErrorHandler.SKIP. +- `description`7 _Optional[Text], optional_ - S3 URL for data files. + Defaults to None. +- `description`8 _Optional[Dict[Text, Text]], optional_ - AWS credentials + with access_key_id and secret_access_key. Defaults to None values. +- `description`9 _Optional[Text], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + +- `license`0 - Response containing: + - status: Current processing status + - asset_id: ID of the created dataset + + +**Raises**: + +- `license`1 - If: + - No input data is provided + - Referenced data asset doesn't exist + - Reserved column names are used + - Data rows are misaligned + - Split configuration is invalid + - Processing or upload fails +- `license`2 - If split configuration is invalid. + diff --git a/docs/api-reference/python/aixplain/factories/file_factory.md b/docs/api-reference/python/aixplain/factories/file_factory.md new file mode 100644 index 00000000..66bcbdc8 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/file_factory.md @@ -0,0 +1,190 @@ +--- +sidebar_label: file_factory +title: aixplain.factories.file_factory +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 20th 2023 +Description: + File Factory Class + +### FileFactory Objects + +```python +class FileFactory() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/file_factory.py#L39) + +Factory class for managing file uploads and storage in the aiXplain platform. + +This class provides functionality for uploading files to S3 storage, +checking storage types, and managing file links. It supports various file +types with different size limits and handles both temporary and permanent +storage. + +#### upload + +```python +@classmethod +def upload(cls, + local_path: Text, + tags: Optional[List[Text]] = None, + license: Optional[License] = None, + is_temp: bool = True, + return_download_link: bool = False) -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/file_factory.py#L49) + +Upload a file to the aiXplain S3 storage. + +This method uploads a file to S3 storage with size limits based on file type: +- Audio: 50MB +- Application: 25MB +- Video: 300MB +- Image: 25MB +- Database: 300MB +- Other: 50MB + +**Arguments**: + +- `local_path` _Text_ - Path to the file to upload. +- `tags` _Optional[List[Text]], optional_ - Tags to associate with the file. + Defaults to None. +- `license` _Optional[License], optional_ - License type for the file. + Required for non-temporary files. Defaults to None. +- `is_temp` _bool, optional_ - Whether this is a temporary upload. + Defaults to True. +- `return_download_link` _bool, optional_ - Whether to return a download + link instead of S3 path. Only valid for temporary files. + Defaults to False. + + +**Returns**: + +- `Text` - Either: + - S3 path where the file was uploaded (if return_download_link=False) + - Download URL for the file (if return_download_link=True) + + +**Raises**: + +- `FileNotFoundError` - If the local file doesn't exist. +- `Exception` - If: + - File size exceeds the type-specific limit + - Requesting download link for non-temporary file +- `AssertionError` - If requesting download link for non-temporary file. + +#### check\_storage\_type + +```python +@classmethod +def check_storage_type(cls, input_link: Any) -> StorageType +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/file_factory.py#L137) + +Determine the storage type of a given input. + +This method checks whether the input is a local file path, a URL +(including S3 and HTTP/HTTPS links), or raw text content. + +**Arguments**: + +- `input_link` _Any_ - Input to check. Can be a file path, URL, or text. + + +**Returns**: + +- `StorageType` - Storage type enum value: + - StorageType.FILE: Local file path + - StorageType.URL: S3 or HTTP/HTTPS URL + - StorageType.TEXT: Raw text content + +#### to\_link + +```python +@classmethod +def to_link(cls, data: Union[Text, Dict], **kwargs) -> Union[Text, Dict] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/file_factory.py#L165) + +Convert local file paths to aiXplain platform links. + +This method checks if the input contains local file paths and uploads +them to the platform, replacing the paths with the resulting URLs. +Other types of input (URLs, text) are left unchanged. + +**Arguments**: + +- `data` _Union[Text, Dict]_ - Input data to process. Can be: + - Text: Single file path, URL, or text content + - Dict: Dictionary with string values that may be file paths +- `**kwargs` - Additional arguments passed to upload() method. + + +**Returns**: + + Union[Text, Dict]: Processed input where any local file paths have + been replaced with platform URLs. Structure matches input type. + +#### create + +```python +@classmethod +def create(cls, + local_path: Text, + tags: Optional[List[Text]] = None, + license: Optional[License] = None, + is_temp: bool = False) -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/file_factory.py#L193) + +Create a permanent or temporary file asset in the platform. + +This method is similar to upload() but with a focus on creating file +assets. For permanent assets (is_temp=False), a license is required. + +**Arguments**: + +- `local_path` _Text_ - Path to the file to upload. +- `tags` _Optional[List[Text]], optional_ - Tags to associate with the file. + Defaults to None. +- `license` _Optional[License], optional_ - License type for the file. + Required for non-temporary files. Defaults to None. +- `is_temp` _bool, optional_ - Whether this is a temporary upload. + Defaults to False. + + +**Returns**: + +- `Text` - Either: + - S3 path for permanent files (is_temp=False) + - Download URL for temporary files (is_temp=True) + + +**Raises**: + +- `FileNotFoundError` - If the local file doesn't exist. +- `Exception` - If file size exceeds the type-specific limit. +- `AssertionError` - If license is not provided for non-temporary files. + diff --git a/docs/api-reference/python/aixplain/factories/finetune_factory/init.md b/docs/api-reference/python/aixplain/factories/finetune_factory/init.md new file mode 100644 index 00000000..07070deb --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/finetune_factory/init.md @@ -0,0 +1,91 @@ +--- +sidebar_label: finetune_factory +title: aixplain.factories.finetune_factory +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: June 14th 2023 +Description: + Finetune Factory Class + +### FinetuneFactory Objects + +```python +class FinetuneFactory() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/finetune_factory/__init__.py#L40) + +Factory class for creating and managing model fine-tuning operations. + +This class provides static methods to create and manage fine-tuning jobs +for machine learning models. It handles cost estimation, dataset preparation, +and fine-tuning configuration. + +**Attributes**: + +- `backend_url` _str_ - Base URL for the aiXplain backend API. + +#### create + +```python +@classmethod +def create(cls, + name: Text, + dataset_list: List[Union[Dataset, Text]], + model: Union[Model, Text], + prompt_template: Optional[Text] = None, + hyperparameters: Optional[Hyperparameters] = None, + train_percentage: Optional[float] = 100, + dev_percentage: Optional[float] = 0) -> Finetune +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/finetune_factory/__init__.py#L68) + +Create a new fine-tuning job with the specified configuration. + +This method sets up a fine-tuning job by validating the configuration, +estimating costs, and preparing the datasets and model. It supports both +direct Dataset/Model objects and their IDs as inputs. + +**Arguments**: + +- `name` _Text_ - Name for the fine-tuning job. +- `dataset_list` _List[Union[Dataset, Text]]_ - List of Dataset objects or dataset IDs + to use for fine-tuning. +- `model` _Union[Model, Text]_ - Model object or model ID to be fine-tuned. +- `prompt_template` _Text, optional_ - Template for formatting training examples. + Use <<COLUMN_NAME>> to reference dataset columns. Defaults to None. +- `hyperparameters` _Hyperparameters, optional_ - Fine-tuning hyperparameters + configuration. Defaults to None. +- `train_percentage` _float, optional_ - Percentage of data to use for training. + Must be > 0. Defaults to 100. +- `dev_percentage` _float, optional_ - Percentage of data to use for validation. + train_percentage + dev_percentage must be <= 100. Defaults to 0. + + +**Returns**: + +- `Finetune` - Configured fine-tuning job object, or None if creation failed. + + +**Raises**: + +- `AssertionError` - If train_percentage <= 0 or train_percentage + dev_percentage > 100. + diff --git a/docs/api-reference/python/aixplain/factories/finetune_factory/prompt_validator.md b/docs/api-reference/python/aixplain/factories/finetune_factory/prompt_validator.md new file mode 100644 index 00000000..cf27fc1b --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/finetune_factory/prompt_validator.md @@ -0,0 +1,40 @@ +--- +sidebar_label: prompt_validator +title: aixplain.factories.finetune_factory.prompt_validator +--- + +#### validate\_prompt + +```python +def validate_prompt(prompt: Text, dataset_list: List[Dataset]) -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/finetune_factory/prompt_validator.py#L23) + +Validate and normalize a prompt template against a list of datasets. + +This function processes a prompt template that contains references to dataset +columns in the format <<COLUMN_NAME>> or <<COLUMN_ID>>. It validates that all +referenced columns exist in the provided datasets and normalizes column IDs +to their corresponding names. + +**Arguments**: + +- `prompt` _Text_ - Prompt template containing column references in + <<COLUMN_NAME>> or <<COLUMN_ID>> format. +- `dataset_list` _List[Dataset]_ - List of datasets to validate the + prompt template against. + + +**Returns**: + +- `Text` - Normalized prompt template with column references converted + to \{COLUMN_NAME} format. + + +**Raises**: + +- `AssertionError` - If any of these conditions are met: + - Multiple datasets have the same referenced column name + - Referenced columns are not found in any dataset + diff --git a/docs/api-reference/python/aixplain/factories/index_factory/init.md b/docs/api-reference/python/aixplain/factories/index_factory/init.md new file mode 100644 index 00000000..480f83f8 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/index_factory/init.md @@ -0,0 +1,149 @@ +--- +sidebar_label: index_factory +title: aixplain.factories.index_factory +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Abdul Basit Anees, Thiago Castro Ferreira, Zaina Abushaban +Date: December 26th 2024 +Description: + Index Factory Class + +#### validate\_embedding\_model + +```python +def validate_embedding_model(model_id: Union[EmbeddingModel, str]) -> bool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/__init__.py#L42) + +Validate that a model is a text embedding model. + +**Arguments**: + +- `model_id` _Union[EmbeddingModel, str]_ - The model ID or EmbeddingModel enum + value to validate. + + +**Returns**: + +- `bool` - True if the model is a text embedding model, False otherwise. + +### IndexFactory Objects + +```python +class IndexFactory(ModelFactory, Generic[T]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/__init__.py#L56) + +Factory class for creating and managing index collections. + +This class extends ModelFactory to provide specialized functionality for +managing index collections, which are used for efficient data retrieval +and searching. It supports various index types through the generic +parameter T. + +**Attributes**: + +- `T` _TypeVar_ - Type variable bound to BaseIndexParams, representing + the specific index parameters type. + +#### create + +```python +@classmethod +def create(cls, + name: Optional[Text] = None, + description: Optional[Text] = None, + embedding_model: Union[EmbeddingModel, + str] = EmbeddingModel.OPENAI_ADA002, + params: Optional[T] = None, + **kwargs) -> IndexModel +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/__init__.py#L70) + +Create a new index collection for efficient data retrieval. + +This method supports two ways of creating an index: +1. Using individual parameters (name, description, embedding_model) - Deprecated +2. Using a params object of type T (recommended) + +**Arguments**: + +- `name` _Optional[Text], optional_ - Name of the index collection. + Deprecated, use params instead. Defaults to None. +- `description` _Optional[Text], optional_ - Description of the index collection. + Deprecated, use params instead. Defaults to None. +- `embedding_model` _Union[EmbeddingModel, str], optional_ - Model to use for text embeddings. + Deprecated, use params instead. Defaults to EmbeddingModel.OPENAI_ADA002. +- `params` _Optional[T], optional_ - Index parameters object. This is the + recommended way to create an index. Defaults to None. +- `**kwargs` - Additional keyword arguments. + + +**Returns**: + +- `IndexModel` - Created index collection model. + + +**Raises**: + +- `AssertionError` - If neither params nor all legacy parameters are provided, + or if both params and legacy parameters are provided. +- `Exception` - If index creation fails. + +#### list + +```python +@classmethod +def list(cls, + query: Optional[Text] = "", + suppliers: Optional[Union[Supplier, List[Supplier]]] = None, + ownership: Optional[Tuple[OwnershipType, List[OwnershipType]]] = None, + sort_by: Optional[SortBy] = None, + sort_order: SortOrder = SortOrder.ASCENDING, + page_number: int = 0, + page_size: int = 20) -> List[IndexModel] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/__init__.py#L142) + +List available index collections with optional filtering and sorting. + +**Arguments**: + +- `query` _Optional[Text], optional_ - Search query to filter indexes. + Defaults to "". +- `suppliers` _Optional[Union[Supplier, List[Supplier]]], optional_ - Filter by + supplier(s). Defaults to None. + ownership (Optional[Tuple[OwnershipType, List[OwnershipType]]], optional): + Filter by ownership type. Defaults to None. +- `sort_by` _Optional[SortBy], optional_ - Field to sort results by. + Defaults to None. +- `sort_order` _SortOrder, optional_ - Sort direction (ascending/descending). + Defaults to SortOrder.ASCENDING. +- `page_number` _int, optional_ - Page number for pagination. Defaults to 0. +- `page_size` _int, optional_ - Number of results per page. Defaults to 20. + + +**Returns**: + +- `List[IndexModel]` - List of index models matching the specified criteria. + diff --git a/docs/api-reference/python/aixplain/factories/index_factory/utils.md b/docs/api-reference/python/aixplain/factories/index_factory/utils.md new file mode 100644 index 00000000..012619a0 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/index_factory/utils.md @@ -0,0 +1,243 @@ +--- +sidebar_label: utils +title: aixplain.factories.index_factory.utils +--- + +### BaseIndexParams Objects + +```python +class BaseIndexParams(BaseModel, ABC) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L9) + +Abstract base class for index parameters. + +This class defines the common parameters and functionality for all index types. +It uses Pydantic for data validation and serialization. + +**Attributes**: + +- `model_config` _ConfigDict_ - Pydantic configuration using enum values. +- `name` _Text_ - Name of the index. +- `description` _Optional[Text]_ - Description of the index. Defaults to "". + +#### to\_dict + +```python +def to_dict() -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L25) + +Convert the parameters to a dictionary format. + +Converts the parameters to a dictionary suitable for API requests, +renaming 'name' to 'data' in the process. + +**Returns**: + +- `Dict` - Dictionary representation of the parameters. + +#### id + +```python +@property +@abstractmethod +def id() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L40) + +Abstract property that must be implemented in subclasses. + +### BaseIndexParamsWithEmbeddingModel Objects + +```python +class BaseIndexParamsWithEmbeddingModel(BaseIndexParams, ABC) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L45) + +Abstract base class for index parameters that require an embedding model. + +This class extends BaseIndexParams to add support for embedding model configuration, +including model selection and embedding size settings. + +**Attributes**: + +- `embedding_model` _Optional[Union[EmbeddingModel, str]]_ - Model to use for text + embeddings. Defaults to EmbeddingModel.OPENAI_ADA002. +- `embedding_size` _Optional[int]_ - Size of the embeddings to generate. + Defaults to None. + +#### validate\_embedding\_model + +```python +@field_validator("embedding_model") +def validate_embedding_model(cls, model_id) -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L62) + +Validate that the provided model is a text embedding model. + +**Arguments**: + +- `model_id` _Union[EmbeddingModel, str]_ - Model ID or enum value to validate. + + +**Returns**: + +- `str` - The validated model ID. + + +**Raises**: + +- `ValueError` - If the model is not a text embedding model. + +#### to\_dict + +```python +def to_dict() -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L80) + +Convert the parameters to a dictionary format. + +Extends the base to_dict method to handle embedding-specific parameters, +renaming fields and restructuring as needed for the API. + +**Returns**: + +- `Dict` - Dictionary representation of the parameters with embedding + configuration properly formatted. + +### VectaraParams Objects + +```python +class VectaraParams(BaseIndexParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L98) + +Parameters for creating a Vectara index. + +This class defines the configuration for Vectara's vector search index. + +**Attributes**: + +- `_id` _ClassVar[str]_ - Static model ID for Vectara index type. + +#### id + +```python +@property +def id() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L110) + +Get the model ID for Vectara index type. + +**Returns**: + +- `str` - The Vectara model ID. + +### ZeroEntropyParams Objects + +```python +class ZeroEntropyParams(BaseIndexParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L119) + +Parameters for creating a Zero Entropy index. + +This class defines the configuration for Zero Entropy's vector search index. + +**Attributes**: + +- `_id` _ClassVar[str]_ - Static model ID for Zero Entropy index type. + +#### id + +```python +@property +def id() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L131) + +Get the model ID for Zero Entropy index type. + +**Returns**: + +- `str` - The Zero Entropy model ID. + +### AirParams Objects + +```python +class AirParams(BaseIndexParamsWithEmbeddingModel) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L140) + +Parameters for creating an AIR (aiXplain Index and Retrieval) index. + +This class defines the configuration for AIR's vector search index, +including embedding model settings. + +**Attributes**: + +- `_id` _ClassVar[str]_ - Static model ID for AIR index type. + +#### id + +```python +@property +def id() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L153) + +Get the model ID for AIR index type. + +**Returns**: + +- `str` - The AIR model ID. + +### GraphRAGParams Objects + +```python +class GraphRAGParams(BaseIndexParamsWithEmbeddingModel) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L162) + +Parameters for creating a GraphRAG (Graph-based Retrieval-Augmented Generation) index. + +This class defines the configuration for GraphRAG's vector search index, +including embedding model and LLM settings. + +**Attributes**: + +- `_id` _ClassVar[str]_ - Static model ID for GraphRAG index type. +- `llm` _Optional[Text]_ - ID of the LLM to use for generation. Defaults to None. + +#### id + +```python +@property +def id() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/index_factory/utils.py#L177) + +Get the model ID for GraphRAG index type. + +**Returns**: + +- `str` - The GraphRAG model ID. + diff --git a/docs/api-reference/python/aixplain/factories/init.md b/docs/api-reference/python/aixplain/factories/init.md new file mode 100644 index 00000000..bedaa2c2 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/init.md @@ -0,0 +1,25 @@ +--- +sidebar_label: factories +title: aixplain.factories +--- + +aiXplain SDK Library. +--- + +aiXplain SDK enables python programmers to add AI functions +to their software. + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + diff --git a/docs/api-reference/python/aixplain/factories/integration_factory.md b/docs/api-reference/python/aixplain/factories/integration_factory.md new file mode 100644 index 00000000..12acd758 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/integration_factory.md @@ -0,0 +1,87 @@ +--- +sidebar_label: integration_factory +title: aixplain.factories.integration_factory +--- + +### IntegrationFactory Objects + +```python +class IntegrationFactory(ModelGetterMixin, ModelListMixin) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/integration_factory.py#L11) + +Factory class for creating and managing Integration models. + +This class provides functionality to get and list Integration models using the backend API. +It inherits from ModelGetterMixin and ModelListMixin to provide model retrieval and listing capabilities. + +**Attributes**: + +- `backend_url` - The URL of the backend API endpoint. + +#### get + +```python +@classmethod +def get(cls, + model_id: Text, + api_key: Optional[Text] = None, + use_cache: bool = False) -> Integration +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/integration_factory.py#L23) + +Retrieves a specific Integration model by its ID. + +**Arguments**: + +- `model_id` _Text_ - The unique identifier of the Integration model. +- `api_key` _Optional[Text], optional_ - API key for authentication. Defaults to None. +- `use_cache` _bool, optional_ - Whether to use cached data. Defaults to False. + + +**Returns**: + +- `Integration` - The retrieved Integration model. + + +**Raises**: + +- `AssertionError` - If the provided ID does not correspond to an Integration model. + +#### list + +```python +@classmethod +def list(cls, + query: Optional[Text] = "", + suppliers: Optional[Union[Supplier, List[Supplier]]] = None, + ownership: Optional[Tuple[OwnershipType, List[OwnershipType]]] = None, + sort_by: Optional[SortBy] = None, + sort_order: SortOrder = SortOrder.ASCENDING, + page_number: int = 0, + page_size: int = 20, + api_key: Optional[Text] = None) -> List[Integration] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/integration_factory.py#L42) + +Lists Integration models based on the provided filters and pagination parameters. + +**Arguments**: + +- `query` _Optional[Text], optional_ - Search query string. Defaults to "". +- `suppliers` _Optional[Union[Supplier, List[Supplier]]], optional_ - Filter by supplier(s). Defaults to None. +- `ownership` _Optional[Tuple[OwnershipType, List[OwnershipType]]], optional_ - Filter by ownership type. Defaults to None. +- `sort_by` _Optional[SortBy], optional_ - Field to sort results by. Defaults to None. +- `sort_order` _SortOrder, optional_ - Sort order (ascending/descending). Defaults to SortOrder.ASCENDING. +- `page_number` _int, optional_ - Page number for pagination. Defaults to 0. +- `page_size` _int, optional_ - Number of items per page. Defaults to 20. +- `api_key` _Optional[Text], optional_ - API key for authentication. Defaults to None. + + +**Returns**: + +- `List[Integration]` - A list of Integration models matching the specified criteria. + diff --git a/docs/api-reference/python/aixplain/factories/metric_factory.md b/docs/api-reference/python/aixplain/factories/metric_factory.md new file mode 100644 index 00000000..e764edee --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/metric_factory.md @@ -0,0 +1,107 @@ +--- +sidebar_label: metric_factory +title: aixplain.factories.metric_factory +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: December 1st 2022 +Description: + Metric Factory Class + +### MetricFactory Objects + +```python +class MetricFactory() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/metric_factory.py#L33) + +A static factory class for creating and managing Metric objects. + +This class provides functionality to create, retrieve, and list Metric objects +through the backend API. It includes methods for fetching individual metrics +by ID and listing metrics with various filtering options. + +**Attributes**: + +- `backend_url` _str_ - The URL endpoint for the backend API. + +#### get + +```python +@classmethod +def get(cls, metric_id: Text) -> Metric +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/metric_factory.py#L67) + +Create a Metric object from a metric ID. + +**Arguments**: + +- `metric_id` _Text_ - The unique identifier of the metric to retrieve. + + +**Returns**: + +- `Metric` - The retrieved Metric object. + + +**Raises**: + +- `Exception` - If the metric creation fails, with status code and error message. + +#### list + +```python +@classmethod +def list(cls, + model_id: Text = None, + is_source_required: Optional[bool] = None, + is_reference_required: Optional[bool] = None, + page_number: int = 0, + page_size: int = 20) -> List[Metric] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/metric_factory.py#L100) + +Get a list of supported metrics based on the given filters. + +**Arguments**: + +- `model_id` _Text, optional_ - ID of model for which metrics are to be used. Defaults to None. +- `is_source_required` _bool, optional_ - Filter metrics that require source input. Defaults to None. +- `is_reference_required` _bool, optional_ - Filter metrics that require reference input. Defaults to None. +- `page_number` _int, optional_ - Page number for pagination. Defaults to 0. +- `page_size` _int, optional_ - Number of items per page. Defaults to 20. + + +**Returns**: + +- `Dict` - A dictionary containing: + - results (List[Metric]): List of filtered metrics + - page_total (int): Number of items in the current page + - page_number (int): Current page number + - total (int): Total number of items matching the filters + + +**Raises**: + +- `Exception` - If there is an error retrieving the metrics list. + diff --git a/docs/api-reference/python/aixplain/factories/model_factory/init.md b/docs/api-reference/python/aixplain/factories/model_factory/init.md new file mode 100644 index 00000000..fcbc6e2d --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/model_factory/init.md @@ -0,0 +1,315 @@ +--- +sidebar_label: model_factory +title: aixplain.factories.model_factory +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: September 1st 2022 +Description: + Model Factory Class + +### ModelFactory Objects + +```python +class ModelFactory(ModelGetterMixin, ModelListMixin) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/__init__.py#L33) + +Factory class for creating, managing, and exploring models. + +This class provides functionality for creating various types of models, +managing model repositories, and interacting with the aiXplain platform's +model-related features. + +**Attributes**: + +- `backend_url` _str_ - Base URL for the aiXplain backend API. + +#### create\_utility\_model + +```python +@classmethod +def create_utility_model(cls, + name: Optional[Text] = None, + code: Union[Text, Callable] = None, + inputs: List[UtilityModelInput] = [], + description: Optional[Text] = None, + output_examples: Text = "", + api_key: Optional[Text] = None) -> UtilityModel +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/__init__.py#L47) + +Create a new utility model for custom functionality. + +This method creates a utility model that can execute custom code or functions +with specified inputs and outputs. + +**Arguments**: + +- `name` _Optional[Text]_ - Name of the utility model. +- `code` _Union[Text, Callable]_ - Python code as string or callable function + implementing the model's functionality. +- `inputs` _List[UtilityModelInput], optional_ - List of input specifications. + Defaults to empty list. +- `description` _Optional[Text], optional_ - Description of what the model does. + Defaults to None. +- `output_examples` _Text, optional_ - Examples of expected outputs. + Defaults to empty string. +- `api_key` _Optional[Text], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + +- `UtilityModel` - Created and registered utility model instance. + + +**Raises**: + +- `Exception` - If model creation fails or validation fails. + +#### list\_host\_machines + +```python +@classmethod +def list_host_machines(cls, api_key: Optional[Text] = None) -> List[Dict] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/__init__.py#L117) + +Lists available hosting machines for model. + +**Arguments**: + +- `api_key` _Text, optional_ - Team API key. Defaults to None. + + +**Returns**: + +- `List[Dict]` - List of dictionaries containing information about + each hosting machine. + +#### list\_gpus + +```python +@classmethod +def list_gpus(cls, api_key: Optional[Text] = None) -> List[List[Text]] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/__init__.py#L143) + +List GPU names on which you can host your language model. + +**Arguments**: + +- `api_key` _Text, optional_ - Team API key. Defaults to None. + + +**Returns**: + +- `List[List[Text]]` - List of all available GPUs and their prices. + +#### list\_functions + +```python +@classmethod +def list_functions(cls, + verbose: Optional[bool] = False, + api_key: Optional[Text] = None) -> List[Dict] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/__init__.py#L168) + +Lists supported model functions on platform. + +**Arguments**: + +- `verbose` _Boolean, optional_ - Set to True if a detailed response + is desired; is otherwise False by default. +- `api_key` _Text, optional_ - Team API key. Defaults to None. + + +**Returns**: + +- `List[Dict]` - List of dictionaries containing information about + each supported function. + +#### create\_asset\_repo + +```python +@classmethod +def create_asset_repo(cls, + name: Text, + description: Text, + function: Text, + source_language: Text, + input_modality: Text, + output_modality: Text, + documentation_url: Optional[Text] = "", + api_key: Optional[Text] = None) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/__init__.py#L208) + +Create a new model repository in the platform. + +This method creates and registers a new model repository, setting up the +necessary infrastructure for model deployment. + +**Arguments**: + +- `name` _Text_ - Name of the model. +- `description` _Text_ - Description of the model's functionality. +- `function` _Text_ - Function name from list_functions() defining model's task. +- `source_language` _Text_ - Language code in ISO 639-1 (2-char) or 639-3 (3-char) format. +- `input_modality` _Text_ - Type of input the model accepts (e.g., text, audio). +- `output_modality` _Text_ - Type of output the model produces (e.g., text, audio). +- `documentation_url` _Optional[Text], optional_ - URL to model documentation. + Defaults to empty string. +- `api_key` _Optional[Text], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + +- `Dict` - Repository creation response containing model ID and other details. + + +**Raises**: + +- `Exception` - If function name is invalid. +- `description`0 - If response status code is not 201. + +#### asset\_repo\_login + +```python +@classmethod +def asset_repo_login(cls, api_key: Optional[Text] = None) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/__init__.py#L284) + +Return login credentials for the image repository that corresponds with +the given API_KEY. + +**Arguments**: + +- `api_key` _Text, optional_ - Team API key. Defaults to None. + + +**Returns**: + +- `Dict` - Backend response + +#### onboard\_model + +```python +@classmethod +def onboard_model(cls, + model_id: Text, + image_tag: Text, + image_hash: Text, + host_machine: Optional[Text] = "", + api_key: Optional[Text] = None) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/__init__.py#L311) + +Onboard a model after its image has been pushed to ECR. + +**Arguments**: + +- `model_id` _Text_ - Model ID obtained from CREATE_ASSET_REPO. +- `image_tag` _Text_ - Image tag to be onboarded. +- `image_hash` _Text_ - Image digest. +- `host_machine` _Text, optional_ - Machine on which to host model. +- `api_key` _Text, optional_ - Team API key. Defaults to None. + +**Returns**: + +- `Dict` - Backend response + +#### deploy\_huggingface\_model + +```python +@classmethod +def deploy_huggingface_model(cls, + name: Text, + hf_repo_id: Text, + revision: Optional[Text] = "", + hf_token: Optional[Text] = "", + api_key: Optional[Text] = None) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/__init__.py#L352) + +Deploy a model from Hugging Face Hub to the aiXplain platform. + +This method handles the deployment of a Hugging Face model, including +authentication and configuration setup. + +**Arguments**: + +- `name` _Text_ - Display name for the deployed model. +- `hf_repo_id` _Text_ - Hugging Face repository ID in 'author/model-name' format. +- `revision` _Optional[Text], optional_ - Specific model revision/commit hash. + Defaults to empty string (latest version). +- `hf_token` _Optional[Text], optional_ - Hugging Face access token for private models. + Defaults to empty string. +- `api_key` _Optional[Text], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + +- `Dict` - Deployment response containing model ID and status information. + +#### get\_huggingface\_model\_status + +```python +@classmethod +def get_huggingface_model_status(cls, + model_id: Text, + api_key: Optional[Text] = None) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/__init__.py#L413) + +Check the deployment status of a Hugging Face model. + +This method retrieves the current status and details of a deployed +Hugging Face model. + +**Arguments**: + +- `model_id` _Text_ - Model ID returned by deploy_huggingface_model. +- `api_key` _Optional[Text], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + +- `Dict` - Status response containing: + - status: Current deployment status + - name: Model name + - id: Model ID + - pricing: Pricing information + diff --git a/docs/api-reference/python/aixplain/factories/model_factory/mixins/init.md b/docs/api-reference/python/aixplain/factories/model_factory/mixins/init.md new file mode 100644 index 00000000..d840aef7 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/model_factory/mixins/init.md @@ -0,0 +1,6 @@ +--- +draft: true +sidebar_label: mixins +title: aixplain.factories.model_factory.mixins +--- + diff --git a/docs/api-reference/python/aixplain/factories/model_factory/mixins/model_getter.md b/docs/api-reference/python/aixplain/factories/model_factory/mixins/model_getter.md new file mode 100644 index 00000000..35c7a411 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/model_factory/mixins/model_getter.md @@ -0,0 +1,53 @@ +--- +sidebar_label: model_getter +title: aixplain.factories.model_factory.mixins.model_getter +--- + +### ModelGetterMixin Objects + +```python +class ModelGetterMixin() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/mixins/model_getter.py#L12) + +Mixin class providing model retrieval functionality. + +This mixin provides methods for retrieving model instances from the backend, +with support for caching to improve performance. + +#### get + +```python +@classmethod +def get(cls, + model_id: Text, + api_key: Optional[Text] = None, + use_cache: bool = False) -> Model +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/mixins/model_getter.py#L19) + +Retrieve a model instance by its ID. + +This method attempts to retrieve a model from the cache if enabled, +falling back to fetching from the backend if necessary. + +**Arguments**: + +- `model_id` _Text_ - ID of the model to retrieve. +- `api_key` _Optional[Text], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. +- `use_cache` _bool, optional_ - Whether to attempt retrieving from cache. + Defaults to False. + + +**Returns**: + +- `Model` - Retrieved model instance. + + +**Raises**: + +- `Exception` - If the model cannot be retrieved or doesn't exist. + diff --git a/docs/api-reference/python/aixplain/factories/model_factory/mixins/model_list.md b/docs/api-reference/python/aixplain/factories/model_factory/mixins/model_list.md new file mode 100644 index 00000000..6ed2a10c --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/model_factory/mixins/model_list.md @@ -0,0 +1,89 @@ +--- +sidebar_label: model_list +title: aixplain.factories.model_factory.mixins.model_list +--- + +### ModelListMixin Objects + +```python +class ModelListMixin() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/mixins/model_list.py#L7) + +Mixin class providing model listing functionality. + +This mixin provides methods for retrieving lists of models with various +filtering and sorting options. + +#### list + +```python +@classmethod +def list(cls, + query: Optional[Text] = "", + function: Optional[Function] = None, + suppliers: Optional[Union[Supplier, List[Supplier]]] = None, + source_languages: Optional[Union[Language, List[Language]]] = None, + target_languages: Optional[Union[Language, List[Language]]] = None, + is_finetunable: Optional[bool] = None, + ownership: Optional[Tuple[OwnershipType, List[OwnershipType]]] = None, + sort_by: Optional[SortBy] = None, + sort_order: SortOrder = SortOrder.ASCENDING, + page_number: int = 0, + page_size: int = 20, + model_ids: Optional[List[Text]] = None, + api_key: Optional[Text] = None) -> dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/mixins/model_list.py#L14) + +List and filter available models with pagination support. + +This method provides comprehensive filtering capabilities for retrieving +models. It supports two modes: +1. Filtering by model IDs (exclusive of other filters) +2. Filtering by various criteria (function, language, etc.) + +**Arguments**: + +- `query` _Optional[Text], optional_ - Search query to filter models. + Defaults to "". +- `function` _Optional[Function], optional_ - Filter by model function/task. + Defaults to None. +- `suppliers` _Optional[Union[Supplier, List[Supplier]]], optional_ - Filter by + supplier(s). Defaults to None. + source_languages (Optional[Union[Language, List[Language]]], optional): + Filter by input language(s). Defaults to None. + target_languages (Optional[Union[Language, List[Language]]], optional): + Filter by output language(s). Defaults to None. +- `is_finetunable` _Optional[bool], optional_ - Filter by fine-tuning capability. + Defaults to None. + ownership (Optional[Tuple[OwnershipType, List[OwnershipType]]], optional): + Filter by ownership type (e.g., SUBSCRIBED, OWNER). Defaults to None. +- `sort_by` _Optional[SortBy], optional_ - Attribute to sort results by. + Defaults to None. +- `sort_order` _SortOrder, optional_ - Sort direction (ascending/descending). + Defaults to SortOrder.ASCENDING. +- `page_number` _int, optional_ - Page number for pagination. Defaults to 0. +- `page_size` _int, optional_ - Number of results per page. Defaults to 20. +- `model_ids` _Optional[List[Text]], optional_ - List of specific model IDs to retrieve. + If provided, other filters are ignored. Defaults to None. +- `api_key` _Optional[Text], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + +- `function`0 - Dictionary containing: + - results (List[Model]): List of models matching the criteria + - page_total (int): Number of models in current page + - page_number (int): Current page number + - total (int): Total number of models matching the criteria + + +**Raises**: + +- `function`1 - If model_ids is provided with other filters, or if + page_size is less than the number of requested model_ids. + diff --git a/docs/api-reference/python/aixplain/factories/model_factory/utils.md b/docs/api-reference/python/aixplain/factories/model_factory/utils.md new file mode 100644 index 00000000..80562490 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/model_factory/utils.md @@ -0,0 +1,135 @@ +--- +sidebar_label: utils +title: aixplain.factories.model_factory.utils +--- + +#### create\_model\_from\_response + +```python +def create_model_from_response(response: Dict) -> Model +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/utils.py#L20) + +Convert API response JSON into appropriate Model object. + +This function creates the correct type of Model object (Model, LLM, IndexModel, +Integration, ConnectionTool, MCPConnection, or UtilityModel) based on the +function type and parameters in the response. + +**Arguments**: + +- `response` _Dict_ - API response containing model information including: + - id: Model identifier + - name: Model name + - function: Function type information + - params: Model parameters + - api_key: Optional API key + - attributes: Optional model attributes + - code: Optional model code + - version: Optional version information + + +**Returns**: + +- `Model` - Instantiated model object of the appropriate subclass based on + the function type. + + +**Raises**: + +- `Exception` - If required code is not found for UtilityModel. + +#### get\_assets\_from\_page + +```python +def get_assets_from_page(query, + page_number: int, + page_size: int, + function: Function, + suppliers: Union[Supplier, List[Supplier]], + source_languages: Union[Language, List[Language]], + target_languages: Union[Language, List[Language]], + is_finetunable: bool = None, + ownership: Optional[Tuple[ + OwnershipType, List[OwnershipType]]] = None, + sort_by: Optional[SortBy] = None, + sort_order: SortOrder = SortOrder.ASCENDING, + api_key: Optional[str] = None) -> List[Model] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/utils.py#L145) + +Retrieve a paginated list of models with specified filters. + +This function fetches a page of models from the aiXplain platform, applying +various filters such as function type, suppliers, languages, and ownership. + +**Arguments**: + +- `query` - Search query string to filter models. +- `page_number` _int_ - Page number to retrieve (0-based). +- `page_size` _int_ - Number of models per page. +- `function` _Function_ - Function type to filter models by. +- `suppliers` _Union[Supplier, List[Supplier]]_ - Single supplier or list of + suppliers to filter models by. +- `source_languages` _Union[Language, List[Language]]_ - Source language(s) + supported by the models. +- `target_languages` _Union[Language, List[Language]]_ - Target language(s) + for translation models. +- `is_finetunable` _bool, optional_ - Filter for fine-tunable models. + Defaults to None. + ownership (Optional[Tuple[OwnershipType, List[OwnershipType]]], optional): + Filter by model ownership type. Defaults to None. +- `sort_by` _Optional[SortBy], optional_ - Field to sort results by. + Defaults to None. +- `sort_order` _SortOrder, optional_ - Sort direction (ascending/descending). + Defaults to SortOrder.ASCENDING. +- `page_number`0 _Optional[str], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + + Tuple[List[Model], int]: A tuple containing: + - List of Model objects matching the filters + - Total number of models matching the filters + + +**Raises**: + +- `page_number`1 - If the API request fails or returns an error. + +#### get\_model\_from\_ids + +```python +def get_model_from_ids(model_ids: List[str], + api_key: Optional[str] = None) -> List[Model] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/model_factory/utils.py#L258) + +Retrieve multiple models by their IDs. + +This function fetches multiple models from the aiXplain platform in a single +request using their unique identifiers. + +**Arguments**: + +- `model_ids` _List[str]_ - List of model IDs to retrieve. +- `api_key` _Optional[str], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + +- `List[Model]` - List of Model objects corresponding to the provided IDs. + Each model will be instantiated as the appropriate subclass based + on its function type. + + +**Raises**: + +- `Exception` - If the API request fails or returns an error, including + cases where models are not found or access is denied. + diff --git a/docs/api-reference/python/aixplain/factories/pipeline_factory/init.md b/docs/api-reference/python/aixplain/factories/pipeline_factory/init.md new file mode 100644 index 00000000..aec42bdd --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/pipeline_factory/init.md @@ -0,0 +1,260 @@ +--- +sidebar_label: pipeline_factory +title: aixplain.factories.pipeline_factory +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: September 1st 2022 +Description: + Pipeline Factory Class + +### PipelineFactory Objects + +```python +class PipelineFactory() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/pipeline_factory/__init__.py#L39) + +Factory class for creating, managing, and exploring pipeline objects. + +This class provides functionality for creating new pipelines, retrieving existing +pipelines, and managing pipeline configurations in the aiXplain platform. + +**Attributes**: + +- `backend_url` _str_ - Base URL for the aiXplain backend API. + +#### get + +```python +@classmethod +def get(cls, pipeline_id: Text, api_key: Optional[Text] = None) -> Pipeline +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/pipeline_factory/__init__.py#L52) + +Retrieve a pipeline by its ID. + +This method fetches an existing pipeline from the aiXplain platform using +its unique identifier. + +**Arguments**: + +- `pipeline_id` _Text_ - Unique identifier of the pipeline to retrieve. +- `api_key` _Optional[Text], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + +- `Pipeline` - Retrieved pipeline object with its configuration and architecture. + + +**Raises**: + +- `Exception` - If the pipeline cannot be retrieved, including cases where: + - Pipeline ID is invalid + - Network error occurs + - Authentication fails + +#### get\_assets\_from\_page + +```python +@classmethod +def get_assets_from_page(cls, page_number: int) -> List[Pipeline] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/pipeline_factory/__init__.py#L125) + +Retrieve a paginated list of pipelines. + +This method fetches a page of pipelines from the aiXplain platform. +Each page contains up to 10 pipelines. + +**Arguments**: + +- `page_number` _int_ - Zero-based page number to retrieve. + + +**Returns**: + +- `List[Pipeline]` - List of pipeline objects on the specified page. + Returns an empty list if an error occurs or no pipelines are found. + + +**Notes**: + + This method is primarily used internally by get_first_k_assets. + For more control over pipeline listing, use the list method instead. + +#### get\_first\_k\_assets + +```python +@classmethod +def get_first_k_assets(cls, k: int) -> List[Pipeline] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/pipeline_factory/__init__.py#L161) + +Retrieve the first K pipelines from the platform. + +This method fetches up to K pipelines by making multiple paginated requests +as needed (10 pipelines per page). + +**Arguments**: + +- `k` _int_ - Number of pipelines to retrieve. Must be positive. + + +**Returns**: + +- `List[Pipeline]` - List of up to K pipeline objects. + Returns an empty list if an error occurs. + + +**Notes**: + + For more control over pipeline listing, use the list method instead. + This method is maintained for backwards compatibility. + +#### list + +```python +@classmethod +def list(cls, + query: Optional[Text] = None, + functions: Optional[Union[Function, List[Function]]] = None, + suppliers: Optional[Union[Supplier, List[Supplier]]] = None, + models: Optional[Union[Model, List[Model]]] = None, + input_data_types: Optional[Union[DataType, List[DataType]]] = None, + output_data_types: Optional[Union[DataType, List[DataType]]] = None, + page_number: int = 0, + page_size: int = 20, + drafts_only: bool = False) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/pipeline_factory/__init__.py#L190) + +List and filter pipelines with pagination support. + +This method provides comprehensive filtering and pagination capabilities +for retrieving pipelines from the aiXplain platform. + +**Arguments**: + +- `query` _Optional[Text], optional_ - Search query to filter pipelines by name + or description. Defaults to None. +- `functions` _Optional[Union[Function, List[Function]]], optional_ - Filter by + function type(s). Defaults to None. +- `suppliers` _Optional[Union[Supplier, List[Supplier]]], optional_ - Filter by + supplier(s). Defaults to None. +- `models` _Optional[Union[Model, List[Model]]], optional_ - Filter by specific + model(s) used in pipelines. Defaults to None. + input_data_types (Optional[Union[DataType, List[DataType]]], optional): + Filter by input data type(s). Defaults to None. + output_data_types (Optional[Union[DataType, List[DataType]]], optional): + Filter by output data type(s). Defaults to None. +- `page_number` _int, optional_ - Zero-based page number. Defaults to 0. +- `page_size` _int, optional_ - Number of items per page (1-100). + Defaults to 20. +- `drafts_only` _bool, optional_ - If True, only return draft pipelines. + Defaults to False. + + +**Returns**: + +- `Dict` - Response containing: + - results (List[Pipeline]): List of pipeline objects + - page_total (int): Total items in current page + - page_number (int): Current page number + - total (int): Total number of items across all pages + + +**Raises**: + +- `Exception` - If the request fails or if page_size is invalid. +- `AssertionError` - If page_size is not between 1 and 100. + +#### init + +```python +@classmethod +def init(cls, name: Text, api_key: Optional[Text] = None) -> Pipeline +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/pipeline_factory/__init__.py#L311) + +Initialize a new empty pipeline. + +This method creates a new pipeline instance with no nodes or links, +ready for configuration. + +**Arguments**: + +- `name` _Text_ - Name of the pipeline. +- `api_key` _Optional[Text], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + +- `Pipeline` - New pipeline instance with empty configuration. + +#### create + +```python +@classmethod +def create(cls, + name: Text, + pipeline: Union[Text, Dict], + api_key: Optional[Text] = None) -> Pipeline +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/pipeline_factory/__init__.py#L337) + +Create a new draft pipeline. + +This method creates a new pipeline in draft status from a configuration +provided either as a Python dictionary or a JSON file. + +**Arguments**: + +- `name` _Text_ - Name of the pipeline. +- `pipeline` _Union[Text, Dict]_ - Pipeline configuration either as: + - Dict: Python dictionary containing nodes and links + - Text: Path to a JSON file containing the configuration +- `api_key` _Optional[Text], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + +- `Pipeline` - Created pipeline instance in draft status. + + +**Raises**: + +- `Exception` - If: + - JSON file path is invalid + - File extension is not .json + - Pipeline creation request fails + - Pipeline configuration is invalid +- `AssertionError` - If the pipeline file doesn't exist or isn't a JSON file. + diff --git a/docs/api-reference/python/aixplain/factories/pipeline_factory/utils.md b/docs/api-reference/python/aixplain/factories/pipeline_factory/utils.md new file mode 100644 index 00000000..9f05a5e9 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/pipeline_factory/utils.md @@ -0,0 +1,46 @@ +--- +sidebar_label: utils +title: aixplain.factories.pipeline_factory.utils +--- + +#### build\_from\_response + +```python +def build_from_response(response: Dict, + load_architecture: bool = False) -> Pipeline +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/pipeline_factory/utils.py#L23) + +Convert API response into a Pipeline object. + +This function creates a Pipeline object from an API response, optionally loading +its full architecture including nodes and links. The architecture can include +various node types like Input, Output, BareAsset, BareMetric, Decision, Router, +Script, BareSegmentor, and BareReconstructor. + +**Arguments**: + +- `response` _Dict_ - API response containing pipeline information including: + - id: Pipeline identifier + - name: Pipeline name + - api_key: Optional API key + - status: Pipeline status (defaults to "draft") + - nodes: Optional list of node configurations + - links: Optional list of link configurations +- `load_architecture` _bool, optional_ - Whether to load the full pipeline + architecture including nodes and links. Defaults to False. + + +**Returns**: + +- `Pipeline` - Instantiated pipeline object. If load_architecture is True, + includes all configured nodes and links. If architecture loading fails, + returns a pipeline with empty nodes and links lists. + + +**Notes**: + + When loading architecture, decision nodes with passthrough parameters are + processed first to ensure proper parameter linking. + diff --git a/docs/api-reference/python/aixplain/factories/script_factory.md b/docs/api-reference/python/aixplain/factories/script_factory.md new file mode 100644 index 00000000..b296b7df --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/script_factory.md @@ -0,0 +1,45 @@ +--- +sidebar_label: script_factory +title: aixplain.factories.script_factory +--- + +### ScriptFactory Objects + +```python +class ScriptFactory() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/script_factory.py#L10) + +A factory class for handling script file operations. + +This class provides functionality for uploading script files to the backend +and managing their metadata. + +#### upload\_script + +```python +@classmethod +def upload_script(cls, script_path: str) -> Tuple[str, str] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/script_factory.py#L17) + +Uploads a script file to the backend and returns its ID and metadata. + +**Arguments**: + +- `script_path` _str_ - The file system path to the script file to be uploaded. + + +**Returns**: + + Tuple[str, str]: A tuple containing: + - file_id (str): The unique identifier assigned to the uploaded file. + - metadata (str): JSON string containing file metadata (name and size). + + +**Raises**: + +- `Exception` - If the upload fails or the file cannot be accessed. + diff --git a/docs/api-reference/python/aixplain/factories/team_agent_factory/init.md b/docs/api-reference/python/aixplain/factories/team_agent_factory/init.md new file mode 100644 index 00000000..f8519895 --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/team_agent_factory/init.md @@ -0,0 +1,190 @@ +--- +sidebar_label: team_agent_factory +title: aixplain.factories.team_agent_factory +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Thiago Castro Ferreira and Lucas Pavanelli +Date: August 15th 2024 +Description: + TeamAgent Factory Class + +### TeamAgentFactory Objects + +```python +class TeamAgentFactory() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/team_agent_factory/__init__.py#L42) + +Factory class for creating and managing team agents. + +This class provides functionality for creating new team agents, retrieving existing +team agents, and managing team agent configurations in the aiXplain platform. +Team agents can be composed of multiple individual agents, LLMs, and inspectors +working together to accomplish complex tasks. + +#### create + +```python +@classmethod +def create(cls, + name: Text, + agents: List[Union[Text, Agent]], + llm_id: Text = "669a63646eb56306647e1091", + llm: Optional[Union[LLM, Text]] = None, + supervisor_llm: Optional[Union[LLM, Text]] = None, + mentalist_llm: Optional[Union[LLM, Text]] = None, + description: Text = "", + api_key: Text = config.TEAM_API_KEY, + supplier: Union[Dict, Text, Supplier, int] = "aiXplain", + version: Optional[Text] = None, + use_mentalist: bool = True, + inspectors: List[Inspector] = [], + inspector_targets: List[Union[InspectorTarget, + Text]] = [InspectorTarget.STEPS], + instructions: Optional[Text] = None, + output_format: Optional[OutputFormat] = None, + expected_output: Optional[Union[BaseModel, Text, dict]] = None, + **kwargs) -> TeamAgent +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/team_agent_factory/__init__.py#L52) + +Create a new team agent in the platform. + +**Arguments**: + +- `name` - The name of the team agent. +- `agents` - A list of agents to be added to the team. +- `llm_id` - The ID of the LLM to be used for the team agent. +- `llm` _Optional[Union[LLM, Text]], optional_ - The LLM to be used for the team agent. +- `supervisor_llm` _Optional[Union[LLM, Text]], optional_ - Main supervisor LLM. Defaults to None. +- `mentalist_llm` _Optional[Union[LLM, Text]], optional_ - LLM for planning. Defaults to None. +- `description` - The description of the team agent to be displayed in the aiXplain platform. +- `api_key` - The API key to be used for the team agent. +- `supplier` - The supplier of the team agent. +- `version` - The version of the team agent. +- `agents`0 - Whether to use the mentalist agent. +- `agents`1 - A list of inspectors to be added to the team. +- `agents`2 - Which stages to be inspected during an execution of the team agent. (steps, output) +- `agents`3 - Whether to use the mentalist and inspector agents. (legacy) +- `agents`4 - The instructions to guide the team agent (i.e. appended in the prompt of the team agent). +- `agents`5 - The output format to be used for the team agent. +- `agents`6 - The expected output to be used for the team agent. + +**Returns**: + + A new team agent instance. + +#### create\_from\_dict + +```python +@classmethod +def create_from_dict(cls, dict: Dict) -> TeamAgent +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/team_agent_factory/__init__.py#L263) + +Create a team agent from a dictionary representation. + +This method instantiates a TeamAgent object from a dictionary containing +the agent's configuration. + +**Arguments**: + +- `dict` _Dict_ - Dictionary containing team agent configuration including: + - id: Team agent identifier + - name: Team agent name + - agents: List of agent configurations + - llm: Optional LLM configuration + - supervisor_llm: Optional supervisor LLM configuration + - mentalist_llm: Optional mentalist LLM configuration + + +**Returns**: + +- `TeamAgent` - Instantiated team agent with validated configuration. + + +**Raises**: + +- `Exception` - If validation fails or required fields are missing. + +#### list + +```python +@classmethod +def list(cls) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/team_agent_factory/__init__.py#L290) + +List all team agents available in the platform. + +This method retrieves all team agents accessible to the current user, +using the configured API key. + +**Returns**: + +- `Dict` - Response containing: + - results (List[TeamAgent]): List of team agent objects + - page_total (int): Total items in current page + - page_number (int): Current page number (always 0) + - total (int): Total number of team agents + + +**Raises**: + +- `Exception` - If the request fails or returns an error, including cases + where authentication fails or the service is unavailable. + +#### get + +```python +@classmethod +def get(cls, agent_id: Text, api_key: Optional[Text] = None) -> TeamAgent +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/team_agent_factory/__init__.py#L337) + +Retrieve a team agent by its ID. + +This method fetches a specific team agent from the platform using its +unique identifier. + +**Arguments**: + +- `agent_id` _Text_ - Unique identifier of the team agent to retrieve. +- `api_key` _Optional[Text], optional_ - API key for authentication. + Defaults to None, using the configured TEAM_API_KEY. + + +**Returns**: + +- `TeamAgent` - Retrieved team agent with its full configuration. + + +**Raises**: + +- `Exception` - If: + - Team agent ID is invalid + - Authentication fails + - Service is unavailable + - Other API errors occur + diff --git a/docs/api-reference/python/aixplain/factories/team_agent_factory/inspector_factory.md b/docs/api-reference/python/aixplain/factories/team_agent_factory/inspector_factory.md new file mode 100644 index 00000000..7b14c9ef --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/team_agent_factory/inspector_factory.md @@ -0,0 +1,126 @@ +--- +sidebar_label: inspector_factory +title: aixplain.factories.team_agent_factory.inspector_factory +--- + +Factory module for creating and configuring inspector agents. + +This module provides functionality for creating inspector agents that can validate +and monitor team agent operations. Inspectors can be created from existing models +or using automatic configurations. + +WARNING: This feature is currently in private beta. + +**Example**: + + Create an inspector from a model with adaptive policy:: + + inspector = InspectorFactory.create_from_model( + name="my_inspector", + model_id="my_model", +- `model_config=\{"prompt"` - "Check if the data is safe to use."}, + policy=InspectorPolicy.ADAPTIVE, + ) + + +**Notes**: + + Currently only supports GUARDRAILS and TEXT_GENERATION models as inspectors. + +### InspectorFactory Objects + +```python +class InspectorFactory() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/team_agent_factory/inspector_factory.py#L39) + +Factory class for creating and configuring inspector agents. + +This class provides methods for creating inspector agents either from existing +models or using automatic configurations. Inspectors are used to validate and +monitor team agent operations, providing feedback and enforcing policies. + +#### create\_from\_model + +```python +@classmethod +def create_from_model( + cls, + name: Text, + model: Union[Text, Model], + model_config: Optional[Dict] = None, + policy: InspectorPolicy = InspectorPolicy.ADAPTIVE) -> Inspector +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/team_agent_factory/inspector_factory.py#L48) + +Create a new inspector agent from an onboarded model. + +This method creates an inspector agent using an existing model that has been +onboarded to the platform. The model must be of a supported function type +(currently GUARDRAILS or TEXT_GENERATION). + +**Arguments**: + +- `name` _Text_ - Name of the inspector agent. +- `model` _Union[Text, Model]_ - Either a Model instance or model ID string + to use for the inspector. +- `model_config` _Optional[Dict], optional_ - Configuration parameters for + the inspector model (e.g., prompts, thresholds). Defaults to None. +- `policy` _InspectorPolicy, optional_ - Action to take upon negative feedback: + - WARN: Log warning but continue execution + - ABORT: Stop execution on negative feedback + - ADAPTIVE: Dynamically decide based on context + Defaults to InspectorPolicy.ADAPTIVE. + + +**Returns**: + +- `Inspector` - Created and configured inspector agent. + + +**Raises**: + +- `ValueError` - If: + - Model ID is invalid + - Model is not onboarded + - Model function is not supported +- `Exception` - If model retrieval fails + +#### create\_auto + +```python +@classmethod +def create_auto( + cls, + auto: InspectorAuto, + name: Optional[Text] = None, + policy: InspectorPolicy = InspectorPolicy.ADAPTIVE) -> Inspector +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/team_agent_factory/inspector_factory.py#L125) + +Create a new inspector agent using automatic configuration. + +This method creates an inspector agent using a pre-configured InspectorAuto +instance, which provides automatic inspection capabilities without requiring +a specific model. + +**Arguments**: + +- `auto` _InspectorAuto_ - Pre-configured automatic inspector instance. +- `name` _Optional[Text], optional_ - Name for the inspector. If not provided, + uses the name from the auto configuration. Defaults to None. +- `policy` _InspectorPolicy, optional_ - Action to take upon negative feedback: + - WARN: Log warning but continue execution + - ABORT: Stop execution on negative feedback + - ADAPTIVE: Dynamically decide based on context + Defaults to InspectorPolicy.ADAPTIVE. + + +**Returns**: + +- `Inspector` - Created and configured inspector agent using automatic + inspection capabilities. + diff --git a/docs/api-reference/python/aixplain/factories/team_agent_factory/utils.md b/docs/api-reference/python/aixplain/factories/team_agent_factory/utils.md new file mode 100644 index 00000000..5bbfd73c --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/team_agent_factory/utils.md @@ -0,0 +1,54 @@ +--- +sidebar_label: utils +title: aixplain.factories.team_agent_factory.utils +--- + +#### build\_team\_agent + +```python +def build_team_agent(payload: Dict, + agents: List[Agent] = None, + api_key: Text = config.TEAM_API_KEY) -> TeamAgent +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/team_agent_factory/utils.py#L20) + +Build a TeamAgent instance from configuration payload. + +This function creates a TeamAgent instance from a configuration payload, +handling the setup of agents, LLMs, inspectors, and task dependencies. + +**Arguments**: + +- `payload` _Dict_ - Configuration dictionary containing: + - id: Optional team agent ID + - name: Team agent name + - agents: List of agent configurations + - description: Optional description + - role: Optional instructions + - teamId: Optional supplier information + - version: Optional version + - cost: Optional cost information + - llmId: LLM model ID (defaults to GPT-4) + - plannerId: Optional planner model ID + - inspectors: Optional list of inspector configurations + - inspectorTargets: Optional list of inspection targets + - status: Team agent status + - tools: Optional list of tool configurations +- `agents` _List[Agent], optional_ - Pre-instantiated agent objects. If not + provided, agents will be instantiated from IDs in the payload. + Defaults to None. +- `api_key` _Text, optional_ - API key for authentication. Defaults to + config.TEAM_API_KEY. + + +**Returns**: + +- `TeamAgent` - Configured team agent instance with all components initialized. + + +**Raises**: + +- `Exception` - If a task dependency referenced in an agent's configuration + cannot be found. + diff --git a/docs/api-reference/python/aixplain/factories/tool_factory.md b/docs/api-reference/python/aixplain/factories/tool_factory.md new file mode 100644 index 00000000..e20afc7a --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/tool_factory.md @@ -0,0 +1,150 @@ +--- +sidebar_label: tool_factory +title: aixplain.factories.tool_factory +--- + +### ToolFactory Objects + +```python +class ToolFactory(ModelGetterMixin, ModelListMixin) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/tool_factory.py#L17) + +A factory class for creating and managing various types of tools including indexes, scripts, and connections. + +This class provides functionality to create and manage different types of tools: +- Script models (utility models) +- Search collections (index models) +- Connectors (integration models) + +The factory inherits from ModelGetterMixin and ModelListMixin to provide model retrieval +and listing capabilities. + +**Attributes**: + +- `backend_url` - The URL endpoint for the backend API. + +#### recreate + +```python +@classmethod +def recreate(cls, + integration: Optional[Union[Text, Model]] = None, + tool: Optional[Union[Text, Model]] = None, + params: Optional[Union[BaseUtilityModelParams, BaseIndexParams, + BaseAuthenticationParams]] = None, + data: Optional[Dict] = None, + **kwargs) -> Model +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/tool_factory.py#L35) + +Recreates a tool based on an existing tool's configuration. + +This method creates a new tool instance using the configuration of an existing tool. +It's useful for creating copies or variations of existing tools. + +**Arguments**: + +- `integration` _Optional[Union[Text, Model]], optional_ - The integration model or its ID. Defaults to None. +- `tool` _Optional[Union[Text, Model]], optional_ - The existing tool model or its ID to recreate from. Defaults to None. + params (Optional[Union[BaseUtilityModelParams, BaseIndexParams, BaseAuthenticationParams]], optional): + Parameters for the new tool. Defaults to None. +- `data` _Optional[Dict], optional_ - Additional data for tool creation. Defaults to None. +- `**kwargs` - Additional keyword arguments passed to the tool creation process. + + +**Returns**: + +- `Model` - The newly created tool model. + +#### create + +```python +@classmethod +def create(cls, + integration: Optional[Union[Text, Model]] = None, + params: Optional[Union[BaseUtilityModelParams, BaseIndexParams, + BaseAuthenticationParams]] = None, + authentication_schema: Optional[AuthenticationSchema] = None, + data: Optional[Dict] = None, + **kwargs) -> Model +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/tool_factory.py#L65) + +Factory method to create indexes, script models and connections + +**Examples**: + + Create a script model (option 1): + Option 1: + from aixplain.modules.model.utility_model import BaseUtilityModelParams + + def add(a: int, b: int) -> int: + return a + b + + params = BaseUtilityModelParams( + name="My Script Model", + description="My Script Model Description", + code=add + ) + tool = ToolFactory.create(params=params) + + Option 2: + def add(a: int, b: int) -> int: + """Add two numbers""" + return a + b + + tool = ToolFactory.create( + name="My Script Model", + code=add + ) + + Create a search collection: + Option 1: + from aixplain.factories.index_factory.utils import AirParams + + params = AirParams( + name="My Search Collection", + description="My Search Collection Description" + ) + tool = ToolFactory.create(params=params) + + Option 2: + from aixplain.enums.index_stores import IndexStores + + tool = ToolFactory.create( + integration=IndexStores.VECTARA.get_model_id(), + name="My Search Collection", + description="My Search Collection Description" + ) + + Create a connector: + Option 1: + from aixplain.modules.model.connector import BearerAuthenticationParams + + params = BearerAuthenticationParams( + connector_id="my_connector_id", + token="my_token", + name="My Connection" + ) + tool = ToolFactory.create(params=params) + + Option 2: + tool = ToolFactory.create( + integration="my_connector_id", + name="My Connection", + token="my_token" + ) + + +**Arguments**: + +- `params` - The parameters for the tool + +**Returns**: + + The created tool + diff --git a/docs/api-reference/python/aixplain/factories/wallet_factory.md b/docs/api-reference/python/aixplain/factories/wallet_factory.md new file mode 100644 index 00000000..057fd6dd --- /dev/null +++ b/docs/api-reference/python/aixplain/factories/wallet_factory.md @@ -0,0 +1,50 @@ +--- +sidebar_label: wallet_factory +title: aixplain.factories.wallet_factory +--- + +### WalletFactory Objects + +```python +class WalletFactory() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/wallet_factory.py#L8) + +A factory class for retrieving wallet information. + +This class provides functionality to fetch wallet details including total +and reserved balance information from the backend API. + +**Attributes**: + +- `backend_url` - The URL endpoint for the backend API. + +#### get + +```python +@classmethod +def get(cls, api_key: Text = config.TEAM_API_KEY) -> Wallet +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/factories/wallet_factory.py#L20) + +Retrieves the current wallet information from the backend. + +This method fetches the wallet details including total balance and reserved balance +using the provided API key. + +**Arguments**: + +- `api_key` _Text, optional_ - The API key for authentication. Defaults to config.TEAM_API_KEY. + + +**Returns**: + +- `Wallet` - A Wallet object containing the total and reserved balance information. + + +**Raises**: + +- `Exception` - If the wallet information cannot be retrieved from the backend. + diff --git a/docs/api-reference/python/aixplain/init.md b/docs/api-reference/python/aixplain/init.md new file mode 100644 index 00000000..dab9520f --- /dev/null +++ b/docs/api-reference/python/aixplain/init.md @@ -0,0 +1,25 @@ +--- +sidebar_label: aixplain +title: aixplain +--- + +aiXplain SDK Library. +--- + +aiXplain SDK enables python programmers to add AI functions +to their software. + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + diff --git a/docs/api-reference/python/aixplain/modules/agent/agent_response.md b/docs/api-reference/python/aixplain/modules/agent/agent_response.md new file mode 100644 index 00000000..08a03828 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/agent/agent_response.md @@ -0,0 +1,147 @@ +--- +sidebar_label: agent_response +title: aixplain.modules.agent.agent_response +--- + +### AgentResponse Objects + +```python +class AgentResponse(ModelResponse) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response.py#L7) + +A response object for agent execution results. + +This class extends ModelResponse to handle agent-specific response data, +including intermediate steps and execution statistics. It provides dictionary-like +access to response data and serialization capabilities. + +**Attributes**: + +- `status` _ResponseStatus_ - The status of the agent execution. +- `data` _Optional[AgentResponseData]_ - Structured data from the agent execution. +- `details` _Optional[Union[Dict, List]]_ - Additional execution details. +- `completed` _bool_ - Whether the execution has completed. +- `error_message` _Text_ - Error message if execution failed. +- `used_credits` _float_ - Number of credits used for execution. +- `run_time` _float_ - Total execution time in seconds. +- `usage` _Optional[Dict]_ - Resource usage information. +- `url` _Optional[Text]_ - URL for asynchronous result polling. + +#### \_\_init\_\_ + +```python +def __init__(status: ResponseStatus = ResponseStatus.FAILED, + data: Optional[AgentResponseData] = None, + details: Optional[Union[Dict, List]] = {}, + completed: bool = False, + error_message: Text = "", + used_credits: float = 0.0, + run_time: float = 0.0, + usage: Optional[Dict] = None, + url: Optional[Text] = None, + **kwargs) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response.py#L25) + +Initialize a new AgentResponse instance. + +**Arguments**: + +- `status` _ResponseStatus, optional_ - The status of the agent execution. + Defaults to ResponseStatus.FAILED. +- `data` _Optional[AgentResponseData], optional_ - Structured data from the + agent execution. Defaults to None. +- `details` _Optional[Union[Dict, List]], optional_ - Additional execution + details. Defaults to \{}. +- `completed` _bool, optional_ - Whether the execution has completed. + Defaults to False. +- `error_message` _Text, optional_ - Error message if execution failed. + Defaults to "". +- `used_credits` _float, optional_ - Number of credits used for execution. + Defaults to 0.0. +- `run_time` _float, optional_ - Total execution time in seconds. + Defaults to 0.0. +- `usage` _Optional[Dict], optional_ - Resource usage information. + Defaults to None. +- `url` _Optional[Text], optional_ - URL for asynchronous result polling. + Defaults to None. +- `**kwargs` - Additional keyword arguments passed to ModelResponse. + +#### \_\_getitem\_\_ + +```python +def __getitem__(key: Text) -> Any +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response.py#L76) + +Get a response attribute using dictionary-style access. + +Overrides the parent class's __getitem__ to handle AgentResponseData +serialization when accessing the 'data' key. + +**Arguments**: + +- `key` _Text_ - The name of the attribute to get. + + +**Returns**: + +- `Any` - The value of the attribute. For 'data' key, returns the + serialized dictionary form. + +#### \_\_setitem\_\_ + +```python +def __setitem__(key: Text, value: Any) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response.py#L93) + +Set a response attribute using dictionary-style access. + +Overrides the parent class's __setitem__ to handle AgentResponseData +deserialization when setting the 'data' key. + +**Arguments**: + +- `key` _Text_ - The name of the attribute to set. +- `value` _Any_ - The value to assign. For 'data' key, can be either a + dictionary or AgentResponseData instance. + +#### to\_dict + +```python +def to_dict() -> Dict[Text, Any] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response.py#L111) + +Convert the response to a dictionary representation. + +Overrides the parent class's to_dict to handle AgentResponseData +serialization in the output dictionary. + +**Returns**: + + Dict[Text, Any]: A dictionary containing all response data, with the + 'data' field containing the serialized AgentResponseData. + +#### \_\_repr\_\_ + +```python +def __repr__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response.py#L125) + +Return a string representation of the response. + +**Returns**: + +- `str` - A string showing all attributes and their values in a readable format, + with the class name changed from ModelResponse to AgentResponse. + diff --git a/docs/api-reference/python/aixplain/modules/agent/agent_response_data.md b/docs/api-reference/python/aixplain/modules/agent/agent_response_data.md new file mode 100644 index 00000000..11ecdade --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/agent/agent_response_data.md @@ -0,0 +1,176 @@ +--- +sidebar_label: agent_response_data +title: aixplain.modules.agent.agent_response_data +--- + +### AgentResponseData Objects + +```python +class AgentResponseData() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response_data.py#L4) + +A container for agent execution response data. + +This class encapsulates the input, output, and execution details of an agent's +response, including intermediate steps and execution statistics. + +**Attributes**: + +- `input` _Optional[Any]_ - The input provided to the agent. +- `output` _Optional[Any]_ - The final output from the agent. +- `session_id` _str_ - Identifier for the conversation session. +- `intermediate_steps` _List[Any]_ - List of steps taken during execution. +- `execution_stats` _Optional[Dict[str, Any]]_ - Statistics about the execution. +- `critiques` _str_ - Any critiques or feedback about the execution. + +#### \_\_init\_\_ + +```python +def __init__(input: Optional[Any] = None, + output: Optional[Any] = None, + session_id: str = "", + intermediate_steps: Optional[List[Any]] = None, + execution_stats: Optional[Dict[str, Any]] = None, + critiques: Optional[str] = None) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response_data.py#L18) + +Initialize a new AgentResponseData instance. + +**Arguments**: + +- `input` _Optional[Any], optional_ - The input provided to the agent. + Defaults to None. +- `output` _Optional[Any], optional_ - The final output from the agent. + Defaults to None. +- `session_id` _str, optional_ - Identifier for the conversation session. + Defaults to "". +- `intermediate_steps` _Optional[List[Any]], optional_ - List of steps taken + during execution. Defaults to None. +- `execution_stats` _Optional[Dict[str, Any]], optional_ - Statistics about + the execution. Defaults to None. +- `critiques` _Optional[str], optional_ - Any critiques or feedback about + the execution. Defaults to None. + +#### from\_dict + +```python +@classmethod +def from_dict(cls, data: Dict[str, Any]) -> "AgentResponseData" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response_data.py#L51) + +Create an AgentResponseData instance from a dictionary. + +**Arguments**: + +- `data` _Dict[str, Any]_ - Dictionary containing response data with keys: + - input: The input provided to the agent + - output: The final output from the agent + - session_id: Identifier for the conversation session + - intermediate_steps: List of steps taken during execution + - executionStats: Statistics about the execution + - critiques: Any critiques or feedback + + +**Returns**: + +- `AgentResponseData` - A new instance populated with the dictionary data. + +#### to\_dict + +```python +def to_dict() -> Dict[str, Any] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response_data.py#L75) + +Convert the response data to a dictionary representation. + +**Returns**: + + Dict[str, Any]: A dictionary containing all response data with keys: + - input: The input provided to the agent + - output: The final output from the agent + - session_id: Identifier for the conversation session + - intermediate_steps: List of steps taken during execution + - executionStats: Statistics about the execution + - execution_stats: Alias for executionStats + - critiques: Any critiques or feedback + +#### \_\_getitem\_\_ + +```python +def __getitem__(key: str) -> Any +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response_data.py#L98) + +Get an attribute value using dictionary-style access. + +**Arguments**: + +- `key` _str_ - The name of the attribute to get. + + +**Returns**: + +- `Any` - The value of the attribute, or None if not found. + +#### \_\_setitem\_\_ + +```python +def __setitem__(key: str, value: Any) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response_data.py#L109) + +Set an attribute value using dictionary-style access. + +**Arguments**: + +- `key` _str_ - The name of the attribute to set. +- `value` _Any_ - The value to assign to the attribute. + + +**Raises**: + +- `KeyError` - If the key is not a valid attribute of the class. + +#### \_\_repr\_\_ + +```python +def __repr__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response_data.py#L124) + +Return a string representation of the response data. + +**Returns**: + +- `str` - A string showing all attributes and their values in a readable format. + +#### \_\_contains\_\_ + +```python +def __contains__(key: Text) -> bool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_response_data.py#L140) + +Check if an attribute exists using 'in' operator. + +**Arguments**: + +- `key` _Text_ - The name of the attribute to check. + + +**Returns**: + +- `bool` - True if the attribute exists and is accessible, False otherwise. + diff --git a/docs/api-reference/python/aixplain/modules/agent/agent_task.md b/docs/api-reference/python/aixplain/modules/agent/agent_task.md new file mode 100644 index 00000000..ff52f8a0 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/agent/agent_task.md @@ -0,0 +1,89 @@ +--- +sidebar_label: agent_task +title: aixplain.modules.agent.agent_task +--- + +### AgentTask Objects + +```python +class AgentTask() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_task.py#L4) + +A task definition for an AI agent to execute. + +This class represents a task that can be assigned to an agent, including its +description, expected output, and any dependencies on other tasks. + +**Attributes**: + +- `name` _Text_ - The unique identifier/name of the task. +- `description` _Text_ - Detailed description of what the task should accomplish. +- `expected_output` _Text_ - Description of the expected output format or content. +- `dependencies` _Optional[List[Union[Text, AgentTask]]]_ - List of tasks or task + names that must be completed before this task. Defaults to None. + +#### \_\_init\_\_ + +```python +def __init__(name: Text, + description: Text, + expected_output: Text, + dependencies: Optional[List[Union[Text, "AgentTask"]]] = None) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_task.py#L17) + +Initialize a new AgentTask instance. + +**Arguments**: + +- `name` _Text_ - The unique identifier/name of the task. +- `description` _Text_ - Detailed description of what the task should accomplish. +- `expected_output` _Text_ - Description of the expected output format or content. +- `dependencies` _Optional[List[Union[Text, AgentTask]]], optional_ - List of + tasks or task names that must be completed before this task. + Defaults to None. + +#### to\_dict + +```python +def to_dict() -> dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_task.py#L39) + +Convert the task to a dictionary representation. + +This method serializes the task data, converting any AgentTask dependencies +to their name strings. + +**Returns**: + +- `dict` - A dictionary containing the task data with keys: + - name: The task name + - description: The task description + - expectedOutput: The expected output description + - dependencies: List of dependency names or None + +#### from\_dict + +```python +@classmethod +def from_dict(cls, data: dict) -> "AgentTask" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/agent_task.py#L66) + +Create an AgentTask instance from a dictionary representation. + +**Arguments**: + +- `data` - Dictionary containing AgentTask parameters + + +**Returns**: + + AgentTask instance + diff --git a/docs/api-reference/python/aixplain/modules/agent/init.md b/docs/api-reference/python/aixplain/modules/agent/init.md new file mode 100644 index 00000000..e8de644a --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/agent/init.md @@ -0,0 +1,323 @@ +--- +sidebar_label: agent +title: aixplain.modules.agent +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Lucas Pavanelli and Thiago Castro Ferreira +Date: May 16th 2024 +Description: + Agentification Class + +### Agent Objects + +```python +class Agent(Model, DeployableMixin[Tool]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/__init__.py#L48) + +An advanced AI system that performs tasks using specialized tools from the aiXplain marketplace. + +This class represents an AI agent that can understand natural language instructions, +use various tools and models, and execute complex tasks. It combines a large language +model (LLM) with specialized tools to provide comprehensive task-solving capabilities. + +**Attributes**: + +- `id` _Text_ - ID of the Agent. +- `name` _Text_ - Name of the Agent. +- `tools` _List[Union[Tool, Model]]_ - Collection of tools and models the Agent can use. +- `description` _Text, optional_ - Detailed description of the Agent's capabilities. + Defaults to "". +- `instructions` _Text_ - System instructions/prompt defining the Agent's behavior. +- `llm_id` _Text_ - ID of the large language model. Defaults to GPT-4o + (6646261c6eb563165658bbb1). +- `llm` _Optional[LLM]_ - The LLM instance used by the Agent. +- `supplier` _Text_ - The provider/creator of the Agent. +- `version` _Text_ - Version identifier of the Agent. +- `status` _AssetStatus_ - Current status of the Agent (DRAFT or ONBOARDED). +- `name`0 _List[AgentTask]_ - List of tasks the Agent can perform. +- `name`1 _str_ - URL endpoint for the backend API. +- `name`2 _str_ - Authentication key for API access. +- `name`3 _Dict, optional_ - Pricing information for using the Agent. Defaults to None. +- `name`4 _bool_ - Whether the Agent's configuration is valid. +- `name`3 _Dict, optional_ - model price. Defaults to None. +- `name`6 _OutputFormat_ - default output format for agent responses. +- `name`7 _Union[BaseModel, Text, dict], optional_ - expected output. Defaults to None. + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text, + description: Text, + instructions: Optional[Text] = None, + tools: List[Union[Tool, Model]] = [], + llm_id: Text = "6646261c6eb563165658bbb1", + llm: Optional[LLM] = None, + api_key: Optional[Text] = config.TEAM_API_KEY, + supplier: Union[Dict, Text, Supplier, int] = "aiXplain", + version: Optional[Text] = None, + cost: Optional[Dict] = None, + status: AssetStatus = AssetStatus.DRAFT, + tasks: List[AgentTask] = [], + output_format: OutputFormat = OutputFormat.TEXT, + expected_output: Optional[Union[BaseModel, Text, dict]] = None, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/__init__.py#L80) + +Initialize a new Agent instance. + +**Arguments**: + +- `id` _Text_ - ID of the Agent. +- `name` _Text_ - Name of the Agent. +- `description` _Text_ - Detailed description of the Agent's capabilities. +- `instructions` _Optional[Text], optional_ - System instructions/prompt defining + the Agent's behavior. Defaults to None. +- `tools` _List[Union[Tool, Model]], optional_ - Collection of tools and models + the Agent can use. Defaults to empty list. +- `llm_id` _Text, optional_ - ID of the large language model. Defaults to GPT-4o + (6646261c6eb563165658bbb1). +- `llm` _Optional[LLM], optional_ - The LLM instance to use. If provided, takes + precedence over llm_id. Defaults to None. +- `api_key` _Optional[Text], optional_ - Authentication key for API access. + Defaults to config.TEAM_API_KEY. +- `supplier` _Union[Dict, Text, Supplier, int], optional_ - The provider/creator + of the Agent. Defaults to "aiXplain". +- `version` _Optional[Text], optional_ - Version identifier. Defaults to None. +- `name`0 _Optional[Dict], optional_ - Pricing information. Defaults to None. +- `name`1 _AssetStatus, optional_ - Current status of the Agent. + Defaults to AssetStatus.DRAFT. +- `name`2 _List[AgentTask], optional_ - List of tasks the Agent can perform. + Defaults to empty list. +- `name`3 _OutputFormat, optional_ - default output format for agent responses. Defaults to OutputFormat.TEXT. +- `name`4 _Union[BaseModel, Text, dict], optional_ - expected output. Defaults to None. +- `name`5 - Additional configuration parameters. + +#### validate + +```python +def validate(raise_exception: bool = False) -> bool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/__init__.py#L194) + +Validate the Agent's configuration and mark its validity status. + +This method runs all validation checks and updates the is_valid flag. +If validation fails, it can either raise an exception or log warnings. + +**Arguments**: + +- `raise_exception` _bool, optional_ - Whether to raise exceptions on validation + failures. If False, failures are logged as warnings. Defaults to False. + + +**Returns**: + +- `bool` - True if validation passed, False otherwise. + + +**Raises**: + +- `Exception` - If validation fails and raise_exception is True. + +#### run + +```python +def run( + data: Optional[Union[Dict, Text]] = None, + query: Optional[Text] = None, + session_id: Optional[Text] = None, + history: Optional[List[Dict]] = None, + name: Text = "model_process", + timeout: float = 300, + parameters: Dict = {}, + wait_time: float = 0.5, + content: Optional[Union[Dict[Text, Text], List[Text]]] = None, + max_tokens: int = 4096, + max_iterations: int = 5, + output_format: Optional[OutputFormat] = None, + expected_output: Optional[Union[BaseModel, Text, dict]] = None +) -> AgentResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/__init__.py#L265) + +Runs an agent call. + +**Arguments**: + +- `data` _Optional[Union[Dict, Text]], optional_ - data to be processed by the agent. Defaults to None. +- `query` _Optional[Text], optional_ - query to be processed by the agent. Defaults to None. +- `session_id` _Optional[Text], optional_ - conversation Session ID. Defaults to None. +- `history` _Optional[List[Dict]], optional_ - chat history (in case session ID is None). Defaults to None. +- `name` _Text, optional_ - ID given to a call. Defaults to "model_process". +- `timeout` _float, optional_ - total polling time. Defaults to 300. +- `parameters` _Dict, optional_ - optional parameters to the model. Defaults to "\{}". +- `wait_time` _float, optional_ - wait time in seconds between polling calls. Defaults to 0.5. +- `content` _Union[Dict[Text, Text], List[Text]], optional_ - Content inputs to be processed according to the query. Defaults to None. +- `max_tokens` _int, optional_ - maximum number of tokens which can be generated by the agent. Defaults to 2048. +- `query`0 _int, optional_ - maximum number of iterations between the agent and the tools. Defaults to 10. +- `query`1 _OutputFormat, optional_ - response format. If not provided, uses the format set during initialization. +- `query`2 _Union[BaseModel, Text, dict], optional_ - expected output. Defaults to None. + +**Returns**: + +- `query`3 - parsed output from model + +#### run\_async + +```python +def run_async( + data: Optional[Union[Dict, Text]] = None, + query: Optional[Text] = None, + session_id: Optional[Text] = None, + history: Optional[List[Dict]] = None, + name: Text = "model_process", + parameters: Dict = {}, + content: Optional[Union[Dict[Text, Text], List[Text]]] = None, + max_tokens: int = 2048, + max_iterations: int = 5, + output_format: Optional[OutputFormat] = None, + expected_output: Optional[Union[BaseModel, Text, dict]] = None +) -> AgentResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/__init__.py#L365) + +Runs asynchronously an agent call. + +**Arguments**: + +- `data` _Optional[Union[Dict, Text]], optional_ - data to be processed by the agent. Defaults to None. +- `query` _Optional[Text], optional_ - query to be processed by the agent. Defaults to None. +- `session_id` _Optional[Text], optional_ - conversation Session ID. Defaults to None. +- `history` _Optional[List[Dict]], optional_ - chat history (in case session ID is None). Defaults to None. +- `name` _Text, optional_ - ID given to a call. Defaults to "model_process". +- `parameters` _Dict, optional_ - optional parameters to the model. Defaults to "\{}". +- `content` _Union[Dict[Text, Text], List[Text]], optional_ - Content inputs to be processed according to the query. Defaults to None. +- `max_tokens` _int, optional_ - maximum number of tokens which can be generated by the agent. Defaults to 2048. +- `max_iterations` _int, optional_ - maximum number of iterations between the agent and the tools. Defaults to 10. +- `output_format` _OutputFormat, optional_ - response format. If not provided, uses the format set during initialization. +- `query`0 _Union[BaseModel, Text, dict], optional_ - expected output. Defaults to None. + +**Returns**: + +- `query`1 - polling URL in response + +#### from\_dict + +```python +@classmethod +def from_dict(cls, data: Dict) -> "Agent" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/__init__.py#L527) + +Create an Agent instance from a dictionary representation. + +**Arguments**: + +- `data` - Dictionary containing Agent parameters + + +**Returns**: + + Agent instance + +#### delete + +```python +def delete() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/__init__.py#L602) + +Delete this Agent from the aiXplain platform. + +This method attempts to delete the Agent. The operation will fail if the +Agent is being used by any team agents. + +**Raises**: + +- `Exception` - If deletion fails, with detailed error messages for different + failure scenarios: + - Agent is in use by accessible team agents (lists team agent IDs) + - Agent is in use by inaccessible team agents + - Other deletion errors (with HTTP status code) + +#### update + +```python +def update() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/__init__.py#L665) + +Update the Agent's configuration on the aiXplain platform. + +This method validates and updates the Agent's configuration. It is deprecated +in favor of the save() method. + +**Raises**: + +- `Exception` - If validation fails or if there are errors during the update. +- `DeprecationWarning` - This method is deprecated, use save() instead. + + +**Notes**: + + This method is deprecated and will be removed in a future version. + Please use save() instead. + +#### save + +```python +def save() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/__init__.py#L712) + +Save the Agent's current configuration to the aiXplain platform. + +This method validates and saves any changes made to the Agent's configuration. +It is the preferred method for updating an Agent's settings. + +**Raises**: + +- `Exception` - If validation fails or if there are errors during the save operation. + +#### \_\_repr\_\_ + +```python +def __repr__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/__init__.py#L723) + +Return a string representation of the Agent. + +**Returns**: + +- `str` - A string in the format "Agent: <name> (id=<id>)". + diff --git a/docs/api-reference/python/aixplain/modules/agent/model_with_params.md b/docs/api-reference/python/aixplain/modules/agent/model_with_params.md new file mode 100644 index 00000000..02574fff --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/agent/model_with_params.md @@ -0,0 +1,95 @@ +--- +sidebar_label: model_with_params +title: aixplain.modules.agent.model_with_params +--- + +A generic class that wraps a model with extra parameters. + +This is an abstract base class that must be extended by specific model wrappers. + +Example usage: + +class MyModel(ModelWithParams): + model_id: Text = "my_model" + extra_param: int = 10 + + @field_validator("extra_param") + def validate_extra_param(cls, v: int) -> int: + if v < 0: + raise ValueError("Extra parameter must be positive") + return v + +### ModelWithParams Objects + +```python +class ModelWithParams(BaseModel, ABC) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/model_with_params.py#L25) + +A generic class that wraps a model with extra parameters. + +The extra parameters are not part of the model's input/output parameters. +This is an abstract base class that must be extended by specific model wrappers. + +**Attributes**: + +- `model_id` - The ID of the model to wrap. + +#### validate\_model\_id + +```python +@field_validator("model_id") +def validate_model_id(cls, v: Text) -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/model_with_params.py#L43) + +Validate the model_id field. + +This validator ensures that the model_id is not empty or whitespace-only. + +**Arguments**: + +- `cls` - The class (automatically provided by pydantic). +- `v` _Text_ - The value to validate. + + +**Returns**: + +- `Text` - The validated model ID. + + +**Raises**: + +- `ValueError` - If the model ID is empty or contains only whitespace. + +#### \_\_new\_\_ + +```python +def __new__(cls, *args, **kwargs) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/model_with_params.py#L62) + +Create a new instance of a ModelWithParams subclass. + +This method prevents direct instantiation of the abstract base class while +allowing subclasses to be instantiated normally. + +**Arguments**: + +- `cls` - The class being instantiated. +- `*args` - Positional arguments for instance creation. +- `**kwargs` - Keyword arguments for instance creation. + + +**Returns**: + +- `ModelWithParams` - A new instance of a ModelWithParams subclass. + + +**Raises**: + +- `TypeError` - If attempting to instantiate ModelWithParams directly. + diff --git a/docs/api-reference/python/aixplain/modules/agent/output_format.md b/docs/api-reference/python/aixplain/modules/agent/output_format.md new file mode 100644 index 00000000..6f660bd6 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/agent/output_format.md @@ -0,0 +1,45 @@ +--- +sidebar_label: output_format +title: aixplain.modules.agent.output_format +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: February 21st 2024 +Description: + Asset Enum + +### OutputFormat Objects + +```python +class OutputFormat(Text, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/output_format.py#L28) + +Enum representing different output formats for AI agent responses. + +This enum defines the possible output formats that can be used by AI agents. +Each format is represented by a string constant. + +**Attributes**: + +- `MARKDOWN` _Text_ - Markdown format for formatted text output. +- `TEXT` _Text_ - Plain text output. +- `JSON` _Text_ - JSON format for structured data output. + diff --git a/docs/api-reference/python/aixplain/modules/agent/tool/custom_python_code_tool.md b/docs/api-reference/python/aixplain/modules/agent/tool/custom_python_code_tool.md new file mode 100644 index 00000000..53070926 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/agent/tool/custom_python_code_tool.md @@ -0,0 +1,138 @@ +--- +sidebar_label: custom_python_code_tool +title: aixplain.modules.agent.tool.custom_python_code_tool +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Lucas Pavanelli and Thiago Castro Ferreira +Date: May 16th 2024 +Description: + Agentification Class + +### CustomPythonCodeTool Objects + +```python +class CustomPythonCodeTool(Tool) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/custom_python_code_tool.py#L31) + +A tool for executing custom Python code in the aiXplain platform. + +This tool allows users to define and execute custom Python functions or code snippets +as part of their workflow. It supports both direct code input and callable functions. + +**Attributes**: + +- `code` _Union[Text, Callable]_ - The Python code to execute, either as a string or callable. +- `id` _str_ - The identifier for the code interpreter model. +- `status` _AssetStatus_ - The current status of the tool (DRAFT or ONBOARDED). + +#### \_\_init\_\_ + +```python +def __init__(code: Union[Text, Callable], + description: Text = "", + name: Optional[Text] = None, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/custom_python_code_tool.py#L43) + +Initialize a new CustomPythonCodeTool instance. + +**Arguments**: + +- `code` _Union[Text, Callable]_ - The Python code to execute, either as a string or callable function. +- `description` _Text, optional_ - Description of what the code does. Defaults to "". +- `name` _Optional[Text], optional_ - Name of the tool. Defaults to None. +- `**additional_info` - Additional keyword arguments for tool configuration. + + +**Notes**: + + If description or name are not provided, they may be automatically extracted + from the code's docstring if available. + +#### to\_dict + +```python +def to_dict() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/custom_python_code_tool.py#L65) + +Convert the tool instance to a dictionary representation. + +**Returns**: + +- `dict` - A dictionary containing the tool's configuration with keys: + - id: The tool's identifier + - name: The tool's name + - description: The tool's description + - type: Always "utility" + - utility: Always "custom_python_code" + - utilityCode: The Python code to execute + +#### validate + +```python +def validate() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/custom_python_code_tool.py#L86) + +Validate the tool's configuration and code. + +This method performs several checks: +1. Parses and validates the Python code if it's not an S3 URL +2. Extracts description and name from code's docstring if not provided +3. Ensures all required fields (description, code, name) are non-empty +4. Verifies the tool status is either DRAFT or ONBOARDED + +**Raises**: + +- `AssertionError` - If any validation check fails, with a descriptive error message. + +#### \_\_repr\_\_ + +```python +def __repr__() -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/custom_python_code_tool.py#L123) + +Return a string representation of the tool. + +**Returns**: + +- `Text` - A string in the format "CustomPythonCodeTool(name=<tool_name>)". + +#### deploy + +```python +def deploy() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/custom_python_code_tool.py#L131) + +Deploy the custom Python code tool. + +This is a placeholder method as custom Python code tools are automatically +deployed when created. + diff --git a/docs/api-reference/python/aixplain/modules/agent/tool/init.md b/docs/api-reference/python/aixplain/modules/agent/tool/init.md new file mode 100644 index 00000000..07adbd16 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/agent/tool/init.md @@ -0,0 +1,119 @@ +--- +sidebar_label: tool +title: aixplain.modules.agent.tool +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Lucas Pavanelli and Thiago Castro Ferreira +Date: May 16th 2024 +Description: + Agentification Class + +### Tool Objects + +```python +class Tool(ABC) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/__init__.py#L29) + +Specialized software or resource designed to assist the AI in executing specific tasks or functions based on user commands. + +**Attributes**: + +- `name` _Text_ - name of the tool +- `description` _Text_ - description of the tool +- `version` _Text_ - version of the tool + +#### \_\_init\_\_ + +```python +def __init__(name: Text, + description: Text, + version: Optional[Text] = None, + api_key: Optional[Text] = config.TEAM_API_KEY, + status: Optional[AssetStatus] = AssetStatus.DRAFT, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/__init__.py#L38) + +Initialize a new Tool instance. + +**Arguments**: + +- `name` _Text_ - The name of the tool. +- `description` _Text_ - A description of the tool's functionality. +- `version` _Optional[Text], optional_ - The version of the tool. Defaults to None. +- `api_key` _Optional[Text], optional_ - The API key for authentication. Defaults to config.TEAM_API_KEY. +- `status` _Optional[AssetStatus], optional_ - The current status of the tool. Defaults to AssetStatus.DRAFT. +- `**additional_info` - Additional keyword arguments for tool configuration. + +#### to\_dict + +```python +def to_dict() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/__init__.py#L64) + +Converts the tool instance to a dictionary representation. + +**Returns**: + +- `dict` - A dictionary containing the tool's attributes and configuration. + + +**Raises**: + +- `NotImplementedError` - This is an abstract method that must be implemented by subclasses. + +#### validate + +```python +def validate() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/__init__.py#L75) + +Validates the tool's configuration and settings. + +This method should check if all required attributes are properly set and +if the tool's configuration is valid. + +**Raises**: + +- `NotImplementedError` - This is an abstract method that must be implemented by subclasses. + +#### deploy + +```python +def deploy() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/__init__.py#L86) + +Deploys the tool to make it available for use. + +This method should handle any necessary setup or deployment steps +required to make the tool operational. + +**Raises**: + +- `NotImplementedError` - This is an abstract method that must be implemented by subclasses. + diff --git a/docs/api-reference/python/aixplain/modules/agent/tool/model_tool.md b/docs/api-reference/python/aixplain/modules/agent/tool/model_tool.md new file mode 100644 index 00000000..771b9f87 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/agent/tool/model_tool.md @@ -0,0 +1,222 @@ +--- +sidebar_label: model_tool +title: aixplain.modules.agent.tool.model_tool +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Lucas Pavanelli and Thiago Castro Ferreira +Date: May 16th 2024 +Description: + Agentification Class + +#### set\_tool\_name + +```python +def set_tool_name(function: Function, + supplier: Supplier = None, + model: Model = None) -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/model_tool.py#L30) + +Sets the name of the tool based on the function, supplier, and model. + +**Arguments**: + +- `function` _Function_ - The function to be used in the tool. +- `supplier` _Supplier_ - The supplier to be used in the tool. +- `model` _Model_ - The model to be used in the tool. + + +**Returns**: + +- `Text` - The name of the tool. + +### ModelTool Objects + +```python +class ModelTool(Tool) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/model_tool.py#L54) + +A tool that wraps AI models to execute specific tasks or functions based on user commands. + +This class provides a standardized interface for working with various AI models, +allowing them to be used as tools in the aiXplain platform. It handles model +configuration, validation, and parameter management. + +**Attributes**: + +- `function` _Optional[Function]_ - The task that the tool performs. +- `supplier` _Optional[Supplier]_ - The preferred supplier to perform the task. +- `model` _Optional[Union[Text, Model]]_ - The model ID or Model instance. +- `model_object` _Optional[Model]_ - The actual Model instance for parameter access. +- `parameters` _Optional[Dict]_ - Configuration parameters for the model. +- `status` _AssetStatus_ - The current status of the tool. + +#### \_\_init\_\_ + +```python +def __init__(function: Optional[Union[Function, Text]] = None, + supplier: Optional[Union[Dict, Supplier]] = None, + model: Optional[Union[Text, Model]] = None, + name: Optional[Text] = None, + description: Text = "", + parameters: Optional[Dict] = None, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/model_tool.py#L70) + +Initialize a new ModelTool instance. + +**Arguments**: + +- `function` _Optional[Union[Function, Text]], optional_ - The task that the tool performs. Can be a Function enum + or a string that will be converted to a Function. Defaults to None. +- `supplier` _Optional[Union[Dict, Supplier]], optional_ - The preferred supplier to perform the task. + Can be a Supplier enum or a dictionary with supplier information. Defaults to None. +- `model` _Optional[Union[Text, Model]], optional_ - The model to use, either as a Model instance + or a model ID string. Defaults to None. +- `name` _Optional[Text], optional_ - The name of the tool. If not provided, will be generated + from function, supplier, and model. Defaults to None. +- `description` _Text, optional_ - A description of the tool's functionality. If not provided, + will be taken from model or function description. Defaults to "". +- `parameters` _Optional[Dict], optional_ - Configuration parameters for the model. Defaults to None. +- `**additional_info` - Additional keyword arguments for tool configuration. + + +**Raises**: + +- `Exception` - If the specified model doesn't exist or is inaccessible. + +#### to\_dict + +```python +def to_dict() -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/model_tool.py#L125) + +Convert the tool instance to a dictionary representation. + +This method handles the conversion of complex attributes like supplier and model +into their serializable forms. + +**Returns**: + +- `Dict` - A dictionary containing the tool's configuration with keys: + - function: The function value or None + - type: Always "model" + - name: The tool's name + - description: The tool's description + - supplier: The supplier code or None + - version: The tool's version or None + - assetId: The model's ID + - parameters: The tool's parameters + - status: The tool's status + +#### validate + +```python +def validate() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/model_tool.py#L164) + +Validates the tool. + +**Notes**: + + - Checks if the tool has a function or model. + - If the function is a string, it converts it to a Function enum. + - Checks if the function is a utility function and if it has an associated model. + - Validates the supplier. + - Validates the model. + - If the description is empty, it sets the description to the function description or the model description. + +#### get\_parameters + +```python +def get_parameters() -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/model_tool.py#L217) + +Get the tool's parameters, either from explicit settings or the model object. + +**Returns**: + +- `Dict` - The tool's parameters. If no explicit parameters were set and a model + object exists with model_params, returns those parameters as a list. + +#### validate\_parameters + +```python +def validate_parameters( + received_parameters: Optional[List[Dict]] = None +) -> Optional[List[Dict]] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/model_tool.py#L252) + +Validates and formats the parameters for the tool. + +**Arguments**: + +- `received_parameters` _Optional[List[Dict]]_ - List of parameter dictionaries in format [\{"name": "param_name", "value": param_value}] + + +**Returns**: + +- `Optional[List[Dict]]` - Validated parameters in the required format + + +**Raises**: + +- `ValueError` - If received parameters don't match the expected parameters from model or function + +#### \_\_repr\_\_ + +```python +def __repr__() -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/model_tool.py#L307) + +Return a string representation of the tool. + +**Returns**: + +- `Text` - A string in the format "ModelTool(name=<name>, function=<function>, + supplier=<supplier>, model=<model>)". + +#### deploy + +```python +def deploy() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/model_tool.py#L318) + +Deploy the model tool. + +This is a placeholder method as model tools are managed through the aiXplain platform +and don't require explicit deployment. + diff --git a/docs/api-reference/python/aixplain/modules/agent/tool/pipeline_tool.md b/docs/api-reference/python/aixplain/modules/agent/tool/pipeline_tool.md new file mode 100644 index 00000000..698a9137 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/agent/tool/pipeline_tool.md @@ -0,0 +1,140 @@ +--- +sidebar_label: pipeline_tool +title: aixplain.modules.agent.tool.pipeline_tool +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Lucas Pavanelli and Thiago Castro Ferreira +Date: May 16th 2024 +Description: + Agentification Class + +### PipelineTool Objects + +```python +class PipelineTool(Tool) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/pipeline_tool.py#L30) + +A tool that wraps aiXplain pipelines to execute complex workflows based on user commands. + +This class provides an interface for using aiXplain pipelines as tools, allowing them +to be integrated into agent workflows. It handles pipeline validation, status management, +and execution. + +**Attributes**: + +- `description` _Text_ - A description of what the pipeline tool does. +- `pipeline` _Union[Text, Pipeline]_ - The pipeline to execute, either as a Pipeline instance + or a pipeline ID string. +- `status` _AssetStatus_ - The current status of the pipeline tool. +- `name` _Text_ - The name of the tool, defaults to pipeline name if not provided. + +#### \_\_init\_\_ + +```python +def __init__(description: Text, + pipeline: Union[Text, Pipeline], + name: Optional[Text] = None, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/pipeline_tool.py#L45) + +Initialize a new PipelineTool instance. + +**Arguments**: + +- `description` _Text_ - A description of what the pipeline tool does. +- `pipeline` _Union[Text, Pipeline]_ - The pipeline to execute, either as a Pipeline instance + or a pipeline ID string. +- `name` _Optional[Text], optional_ - The name of the tool. If not provided, will use + the pipeline's name. Defaults to None. +- `**additional_info` - Additional keyword arguments for tool configuration. + + +**Raises**: + +- `Exception` - If the specified pipeline doesn't exist or is inaccessible. + +#### to\_dict + +```python +def to_dict() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/pipeline_tool.py#L73) + +Convert the tool instance to a dictionary representation. + +**Returns**: + +- `dict` - A dictionary containing the tool's configuration with keys: + - assetId: The pipeline ID + - name: The tool's name + - description: The tool's description + - type: Always "pipeline" + - status: The tool's status + +#### \_\_repr\_\_ + +```python +def __repr__() -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/pipeline_tool.py#L92) + +Return a string representation of the tool. + +**Returns**: + +- `Text` - A string in the format "PipelineTool(name=<name>, pipeline=<pipeline>)". + +#### validate + +```python +def validate() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/pipeline_tool.py#L100) + +Validate the pipeline tool's configuration. + +This method performs several checks: +1. Verifies the pipeline exists and is accessible +2. Sets the tool name to the pipeline name if not provided +3. Updates the tool status to match the pipeline status + +**Raises**: + +- `Exception` - If the pipeline doesn't exist or is inaccessible. + +#### deploy + +```python +def deploy() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/pipeline_tool.py#L127) + +Deploy the pipeline tool. + +This is a placeholder method as pipeline tools are managed through the aiXplain platform +and don't require explicit deployment. + diff --git a/docs/api-reference/python/aixplain/modules/agent/tool/python_interpreter_tool.md b/docs/api-reference/python/aixplain/modules/agent/tool/python_interpreter_tool.md new file mode 100644 index 00000000..83ad430d --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/agent/tool/python_interpreter_tool.md @@ -0,0 +1,120 @@ +--- +sidebar_label: python_interpreter_tool +title: aixplain.modules.agent.tool.python_interpreter_tool +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Lucas Pavanelli and Thiago Castro Ferreira +Date: May 16th 2024 +Description: + Agentification Class + +### PythonInterpreterTool Objects + +```python +class PythonInterpreterTool(Tool) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/python_interpreter_tool.py#L30) + +A tool that provides a Python shell for executing Python commands. + +This tool allows direct execution of Python code within the aiXplain platform. +It acts as an interface to a Python interpreter, enabling dynamic code execution +and computation. + +**Attributes**: + +- `name` _Text_ - Always set to "Python Interpreter". +- `description` _Text_ - Description of the tool's functionality. +- `status` _AssetStatus_ - The current status of the tool (ONBOARDED or DRAFT). + +#### \_\_init\_\_ + +```python +def __init__(**additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/python_interpreter_tool.py#L43) + +Initialize a new PythonInterpreterTool instance. + +This initializes a Python interpreter tool with a fixed name and description. +The tool is set to ONBOARDED status by default. + +**Arguments**: + +- `**additional_info` - Additional keyword arguments for tool configuration. + +#### to\_dict + +```python +def to_dict() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/python_interpreter_tool.py#L56) + +Convert the tool instance to a dictionary representation. + +**Returns**: + +- `dict` - A dictionary containing the tool's configuration with keys: + - description: The tool's description + - type: Always "utility" + - utility: Always "custom_python_code" + +#### validate + +```python +def validate() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/python_interpreter_tool.py#L71) + +Validate the tool's configuration. + +This is a placeholder method as the Python interpreter tool has a fixed +configuration that doesn't require validation. + +#### \_\_repr\_\_ + +```python +def __repr__() -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/python_interpreter_tool.py#L79) + +Return a string representation of the tool. + +**Returns**: + +- `Text` - A string in the format "PythonInterpreterTool()". + +#### deploy + +```python +def deploy() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/python_interpreter_tool.py#L87) + +Deploy the Python interpreter tool. + +This is a placeholder method as the Python interpreter tool is automatically +available and doesn't require explicit deployment. + diff --git a/docs/api-reference/python/aixplain/modules/agent/tool/sql_tool.md b/docs/api-reference/python/aixplain/modules/agent/tool/sql_tool.md new file mode 100644 index 00000000..200a0e6a --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/agent/tool/sql_tool.md @@ -0,0 +1,354 @@ +--- +sidebar_label: sql_tool +title: aixplain.modules.agent.tool.sql_tool +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Lucas Pavanelli and Thiago Castro Ferreira +Date: May 16th 2024 +Description: + Agentification Class + +### SQLToolError Objects + +```python +class SQLToolError(Exception) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L34) + +Base exception for SQL Tool errors + +### CSVError Objects + +```python +class CSVError(SQLToolError) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L40) + +Exception for CSV-related errors + +### DatabaseError Objects + +```python +class DatabaseError(SQLToolError) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L46) + +Exception for database-related errors + +#### clean\_column\_name + +```python +def clean_column_name(col: Text) -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L52) + +Clean column names by replacing spaces and special characters with underscores. + +This function makes column names SQLite-compatible by: +1. Converting to lowercase +2. Replacing special characters with underscores +3. Removing duplicate underscores +4. Adding 'col_' prefix to names starting with numbers + +**Arguments**: + +- `col` _Text_ - The original column name. + + +**Returns**: + +- `Text` - The cleaned, SQLite-compatible column name. + +#### check\_duplicate\_columns + +```python +def check_duplicate_columns(df: pd.DataFrame) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L83) + +Check for duplicate column names in DataFrame after cleaning. + +This function checks if any column names would become duplicates after being +cleaned for SQLite compatibility. + +**Arguments**: + +- `df` _pd.DataFrame_ - The DataFrame to check for duplicate column names. + + +**Raises**: + +- `CSVError` - If any cleaned column names would be duplicates. + +#### infer\_sqlite\_type + +```python +def infer_sqlite_type(dtype) -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L113) + +Infer SQLite type from pandas dtype. + +This function maps pandas data types to appropriate SQLite types: +- Integer types -> INTEGER +- Float types -> REAL +- Boolean types -> INTEGER +- Datetime types -> TIMESTAMP +- All others -> TEXT + +**Arguments**: + +- `dtype` - The pandas dtype to convert. + + +**Returns**: + +- `Text` - The corresponding SQLite type. + + +**Notes**: + + Issues a warning when falling back to TEXT type. + +#### get\_table\_schema + +```python +def get_table_schema(database_path: str) -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L145) + +Get the schema of all tables in the SQLite database. + +This function retrieves the CREATE TABLE statements for all tables in the database. + +**Arguments**: + +- `database_path` _str_ - Path to the SQLite database file. + + +**Returns**: + +- `str` - A string containing all table schemas, separated by newlines. + + +**Raises**: + +- `DatabaseError` - If the database file doesn't exist or there's an error accessing it. + + +**Notes**: + + Issues a warning if no tables are found in the database. + +#### create\_database\_from\_csv + +```python +def create_database_from_csv(csv_path: str, + database_path: str, + table_name: str = None) -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L185) + +Create SQLite database from CSV file and return the schema. + +This function creates or modifies a SQLite database by importing data from a CSV file. +It handles column name cleaning, data type inference, and data conversion. + +**Arguments**: + +- `csv_path` _str_ - Path to the CSV file to import. +- `database_path` _str_ - Path where the SQLite database should be created/modified. +- `table_name` _str, optional_ - Name for the table to create. If not provided, + uses the CSV filename (cleaned). Defaults to None. + + +**Returns**: + +- `str` - The schema of the created database. + + +**Raises**: + +- `CSVError` - If there are issues with the CSV file (doesn't exist, empty, parsing error). +- `DatabaseError` - If there are issues with database creation or modification. + + +**Notes**: + + - Issues warnings for column name changes and existing database/table modifications. + - Automatically cleans column names for SQLite compatibility. + - Handles NULL values, timestamps, and numeric data types appropriately. + +#### get\_table\_names\_from\_schema + +```python +def get_table_names_from_schema(schema: str) -> List[str] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L311) + +Extract table names from a database schema string. + +This function parses CREATE TABLE statements to extract table names. + +**Arguments**: + +- `schema` _str_ - The database schema string containing CREATE TABLE statements. + + +**Returns**: + +- `List[str]` - A list of table names found in the schema. Returns an empty list + if no tables are found or if the schema is empty. + +### SQLTool Objects + +```python +class SQLTool(Tool) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L336) + +A tool for executing SQL commands in an SQLite database. + +This tool provides an interface for interacting with SQLite databases, including +executing queries, managing schema, and handling table operations. It supports +both read-only and write operations based on configuration. + +**Attributes**: + +- `description` _Text_ - A description of what the SQL tool does. +- `database` _Text_ - The database URI or path. +- `schema` _Text_ - The database schema containing table definitions. +- `tables` _Optional[Union[List[Text], Text]]_ - List of table names that can be + accessed by this tool. If None, all tables are accessible. +- `enable_commit` _bool_ - Whether write operations (INSERT, UPDATE, DELETE) are + allowed. If False, only read operations are permitted. +- `status` _AssetStatus_ - The current status of the tool (DRAFT or ONBOARDED). + +#### \_\_init\_\_ + +```python +def __init__(name: Text, + description: Text, + database: Text, + schema: Optional[Text] = None, + tables: Optional[Union[List[Text], Text]] = None, + enable_commit: bool = False, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L354) + +Initialize a new SQLTool instance. + +**Arguments**: + +- `name` _Text_ - The name of the tool. +- `description` _Text_ - A description of what the SQL tool does. +- `database` _Text_ - The database URI or path. Can be a local file path, + S3 URI, or HTTP(S) URL. +- `schema` _Optional[Text], optional_ - The database schema containing table + definitions. If not provided, will be inferred from the database. + Defaults to None. +- `tables` _Optional[Union[List[Text], Text]], optional_ - List of table names + that can be accessed by this tool. If not provided, all tables are + accessible. Defaults to None. +- `enable_commit` _bool, optional_ - Whether write operations are allowed. + If False, only read operations are permitted. Defaults to False. +- `**additional_info` - Additional keyword arguments for tool configuration. + + +**Raises**: + +- `SQLToolError` - If required parameters are missing or invalid. + +#### to\_dict + +```python +def to_dict() -> Dict[str, Text] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L394) + +Convert the tool instance to a dictionary representation. + +**Returns**: + + Dict[str, Text]: A dictionary containing the tool's configuration with keys: + - name: The tool's name + - description: The tool's description + - parameters: List of parameter dictionaries containing: + - database: The database URI or path + - schema: The database schema + - tables: Comma-separated list of table names or None + - enable_commit: Whether write operations are allowed + - type: Always "sql" + +#### validate + +```python +def validate() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L420) + +Validate the SQL tool's configuration. + +This method performs several checks: +1. Verifies required fields (description, database) are provided +2. Validates database path/URI format +3. Infers schema from database if not provided +4. Sets table list from schema if not provided +5. Uploads local database file to storage + +**Raises**: + +- `SQLToolError` - If any validation check fails or if there are issues with + database access or file operations. + +#### deploy + +```python +def deploy() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/tool/sql_tool.py#L472) + +Deploy the SQL tool by downloading and preparing the database. + +This method handles the deployment process: +1. For HTTP(S) URLs: Downloads the database file +2. Creates a unique local filename +3. Uploads the database to the aiXplain platform +4. Cleans up temporary files + +**Raises**: + +- `requests.exceptions.RequestException` - If downloading the database fails. +- `Exception` - If there are issues with file operations or uploads. + diff --git a/docs/api-reference/python/aixplain/modules/agent/utils.md b/docs/api-reference/python/aixplain/modules/agent/utils.md new file mode 100644 index 00000000..659c8e52 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/agent/utils.md @@ -0,0 +1,49 @@ +--- +sidebar_label: utils +title: aixplain.modules.agent.utils +--- + +#### process\_variables + +```python +def process_variables(query: Union[Text, Dict], data: Union[Dict, Text], + parameters: Dict, + agent_description: Union[Text, None]) -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/utils.py#L5) + +Process variables in an agent's description and input data. + +This function validates and processes variables in an agent's description and +input data, ensuring that all required variables are present and properly +formatted. + +**Arguments**: + +- `query` _Union[Text, Dict]_ - The input data provided to the agent. +- `data` _Union[Dict, Text]_ - The data to be processed. +- `parameters` _Dict_ - The parameters available to the agent. +- `agent_description` _Union[Text, None]_ - The description of the agent. + + +**Returns**: + +- `Text` - The processed input data with all required variables included. + + +**Raises**: + +- `AssertionError` - If a required variable is not found in the data or parameters. + +#### validate\_history + +```python +def validate_history(history) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/agent/utils.py#L55) + +Validates that `history` is a list of dicts, each with 'role' and 'content' keys. +Raises a ValueError if validation fails. + diff --git a/docs/api-reference/python/aixplain/modules/api_key.md b/docs/api-reference/python/aixplain/modules/api_key.md new file mode 100644 index 00000000..285ca67f --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/api_key.md @@ -0,0 +1,402 @@ +--- +sidebar_label: api_key +title: aixplain.modules.api_key +--- + +### APIKeyLimits Objects + +```python +class APIKeyLimits() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L9) + +Rate limits configuration for an API key. + +This class defines the rate limits that can be applied either globally +to an API key or specifically to a model. + +**Attributes**: + +- `token_per_minute` _int_ - Maximum number of tokens allowed per minute. +- `token_per_day` _int_ - Maximum number of tokens allowed per day. +- `request_per_minute` _int_ - Maximum number of requests allowed per minute. +- `request_per_day` _int_ - Maximum number of requests allowed per day. +- `model` _Optional[Model]_ - The model these limits apply to, if any. + +#### \_\_init\_\_ + +```python +def __init__(token_per_minute: int, + token_per_day: int, + request_per_minute: int, + request_per_day: int, + model: Optional[Union[Text, Model]] = None) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L23) + +Initialize an APIKeyLimits instance. + +**Arguments**: + +- `token_per_minute` _int_ - Maximum number of tokens per minute. +- `token_per_day` _int_ - Maximum number of tokens per day. +- `request_per_minute` _int_ - Maximum number of requests per minute. +- `request_per_day` _int_ - Maximum number of requests per day. +- `model` _Optional[Union[Text, Model]], optional_ - The model to apply + limits to. Can be a model ID or Model instance. Defaults to None. + +### APIKeyUsageLimit Objects + +```python +class APIKeyUsageLimit() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L52) + +#### \_\_init\_\_ + +```python +def __init__(daily_request_count: int, + daily_request_limit: int, + daily_token_count: int, + daily_token_limit: int, + model: Optional[Union[Text, Model]] = None) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L53) + +Get the usage limits of an API key globally (model equals to None) or for a specific model. + +**Arguments**: + +- `daily_request_count` _int_ - number of requests made +- `daily_request_limit` _int_ - limit of requests +- `daily_token_count` _int_ - number of tokens used +- `daily_token_limit` _int_ - limit of tokens +- `model` _Optional[Union[Text, Model]], optional_ - Model which the limits apply. Defaults to None. + +### APIKey Objects + +```python +class APIKey() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L80) + +An API key for accessing aiXplain services. + +This class represents an API key with its associated limits, budget, +and access controls. It can have both global rate limits and +model-specific rate limits. + +**Attributes**: + +- `id` _int_ - The ID of this API key. +- `name` _Text_ - A descriptive name for the API key. +- `budget` _Optional[float]_ - Maximum spending limit, if any. +- `global_limits` _Optional[APIKeyLimits]_ - Rate limits applied globally. +- `asset_limits` _List[APIKeyLimits]_ - Rate limits for specific models. +- `expires_at` _Optional[datetime]_ - Expiration date and time. +- `access_key` _Optional[Text]_ - The actual API key value. +- `is_admin` _bool_ - Whether this is an admin API key. + +#### \_\_init\_\_ + +```python +def __init__(name: Text, + expires_at: Optional[Union[datetime, Text]] = None, + budget: Optional[float] = None, + asset_limits: List[APIKeyLimits] = [], + global_limits: Optional[Union[Dict, APIKeyLimits]] = None, + id: int = "", + access_key: Optional[Text] = None, + is_admin: bool = False) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L98) + +Initialize an APIKey instance. + +**Arguments**: + +- `name` _Text_ - A descriptive name for the API key. +- `expires_at` _Optional[Union[datetime, Text]], optional_ - When the key + expires. Can be a datetime or ISO format string. Defaults to None. +- `budget` _Optional[float], optional_ - Maximum spending limit. + Defaults to None. +- `asset_limits` _List[APIKeyLimits], optional_ - Rate limits for specific + models. Defaults to empty list. +- `global_limits` _Optional[Union[Dict, APIKeyLimits]], optional_ - Global + rate limits. Can be a dict with tpm/tpd/rpm/rpd keys or an + APIKeyLimits instance. Defaults to None. +- `id` _int, optional_ - Unique identifier. Defaults to empty string. +- `access_key` _Optional[Text], optional_ - The actual API key value. + Defaults to None. +- `is_admin` _bool, optional_ - Whether this is an admin key. + Defaults to False. + + +**Notes**: + + The global_limits dict format should have these keys: + - tpm: tokens per minute + - tpd: tokens per day + - rpm: requests per minute + - rpd: requests per day + +#### validate + +```python +def validate() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L161) + +Validate the APIKey configuration. + +This method checks that all rate limits are non-negative and that +referenced models exist and are valid. + +**Raises**: + +- `AssertionError` - If any of these conditions are not met: + - Budget is negative + - Global rate limits are negative + - Asset-specific rate limits are negative +- `Exception` - If a referenced model ID is not a valid aiXplain model. + + +**Notes**: + + - For asset limits, both the model reference and limits are checked + - Models can be specified by ID or Model instance + - Model IDs are resolved to Model instances during validation + +#### to\_dict + +```python +def to_dict() -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L201) + +Convert the APIKey instance to a dictionary representation. + +This method serializes the APIKey and its associated limits into a +format suitable for API requests or storage. + +**Returns**: + +- `Dict` - A dictionary containing: + - id (int): The API key's ID + - name (Text): The API key's name + - budget (Optional[float]): The spending limit + - assetsLimits (List[Dict]): Model-specific limits with: + - tpm: tokens per minute + - tpd: tokens per day + - rpm: requests per minute + - rpd: requests per day + - assetId: model ID + - expiresAt (Optional[Text]): ISO format expiration date + - globalLimits (Optional[Dict]): Global limits with tpm/tpd/rpm/rpd + + +**Notes**: + + - Datetime objects are converted to ISO format strings + - Model instances are referenced by their ID + +#### delete + +```python +def delete() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L256) + +Delete this API key from the system. + +This method permanently removes the API key from the aiXplain platform. +The operation cannot be undone. + +**Raises**: + +- `Exception` - If deletion fails, which can happen if: + - The API key doesn't exist + - The user doesn't have permission to delete it + - The API request fails + - The server returns a non-200 status code + + +**Notes**: + + - This operation is permanent and cannot be undone + - Only the API key owner can delete it + - Uses the team API key for authentication + +#### get\_usage + +```python +def get_usage(asset_id: Optional[Text] = None) -> APIKeyUsageLimit +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L286) + +Get current usage statistics for this API key. + +This method retrieves the current usage counts and limits for the API key, +either globally or for a specific model. + +**Arguments**: + +- `asset_id` _Optional[Text], optional_ - The model ID to get usage for. + If None, returns usage for all models. Defaults to None. + + +**Returns**: + +- `APIKeyUsageLimit` - A list of usage statistics objects containing: + - daily_request_count: Number of requests made today + - daily_request_limit: Maximum requests allowed per day + - daily_token_count: Number of tokens used today + - daily_token_limit: Maximum tokens allowed per day + - model: The model ID these stats apply to (None for global) + + +**Raises**: + +- `Exception` - If the request fails, which can happen if: + - The API key doesn't exist + - The user doesn't have permission to view usage + - The API request fails + - The server returns an error response + + +**Notes**: + + - Uses the team API key for authentication + - Counts reset at the start of each day + - Filtered by asset_id if provided + +#### set\_token\_per\_day + +```python +def set_token_per_day(token_per_day: int, + model: Optional[Union[Text, Model]] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L377) + +Set the daily token limit for this API key. + +**Arguments**: + +- `token_per_day` _int_ - Maximum number of tokens allowed per day. +- `model` _Optional[Union[Text, Model]], optional_ - The model to set + limit for. If None, sets global limit. Defaults to None. + + +**Raises**: + +- `Exception` - If the model isn't configured in this API key's + asset_limits. + + +**Notes**: + + - Model can be specified by ID or Model instance + - For global limits, model should be None + - The new limit takes effect immediately + +#### set\_token\_per\_minute + +```python +def set_token_per_minute(token_per_minute: int, + model: Optional[Union[Text, Model]] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L396) + +Set the per-minute token limit for this API key. + +**Arguments**: + +- `token_per_minute` _int_ - Maximum number of tokens allowed per minute. +- `model` _Optional[Union[Text, Model]], optional_ - The model to set + limit for. If None, sets global limit. Defaults to None. + + +**Raises**: + +- `Exception` - If the model isn't configured in this API key's + asset_limits. + + +**Notes**: + + - Model can be specified by ID or Model instance + - For global limits, model should be None + - The new limit takes effect immediately + +#### set\_request\_per\_day + +```python +def set_request_per_day(request_per_day: int, + model: Optional[Union[Text, Model]] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L415) + +Set the daily request limit for this API key. + +**Arguments**: + +- `request_per_day` _int_ - Maximum number of requests allowed per day. +- `model` _Optional[Union[Text, Model]], optional_ - The model to set + limit for. If None, sets global limit. Defaults to None. + + +**Raises**: + +- `Exception` - If the model isn't configured in this API key's + asset_limits. + + +**Notes**: + + - Model can be specified by ID or Model instance + - For global limits, model should be None + - The new limit takes effect immediately + +#### set\_request\_per\_minute + +```python +def set_request_per_minute(request_per_minute: int, + model: Optional[Union[Text, Model]] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/api_key.py#L434) + +Set the per-minute request limit for this API key. + +**Arguments**: + +- `request_per_minute` _int_ - Maximum number of requests allowed per minute. +- `model` _Optional[Union[Text, Model]], optional_ - The model to set + limit for. If None, sets global limit. Defaults to None. + + +**Raises**: + +- `Exception` - If the model isn't configured in this API key's + asset_limits. + + +**Notes**: + + - Model can be specified by ID or Model instance + - For global limits, model should be None + - The new limit takes effect immediately + diff --git a/docs/api-reference/python/aixplain/modules/asset.md b/docs/api-reference/python/aixplain/modules/asset.md new file mode 100644 index 00000000..585d112a --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/asset.md @@ -0,0 +1,99 @@ +--- +sidebar_label: asset +title: aixplain.modules.asset +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: December 27th 2022 +Description: + Asset Class + +### Asset Objects + +```python +class Asset() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/asset.py#L29) + +A class representing an aiXplain Asset. + +This class provides functionality to create and manage assets in the aiXplain platform. +Assets can be models, datasets, or other resources with associated metadata like +supplier information, version, license, privacy settings, and cost. + +**Attributes**: + +- `id` _Text_ - The unique identifier of the asset. +- `name` _Text_ - The name of the asset. +- `description` _Text_ - A detailed description of the asset. +- `supplier` _Union[Dict, Text, Supplier, int]_ - The supplier of the asset. +- `version` _Text_ - The version of the asset. +- `license` _Optional[License]_ - The license associated with the asset. +- `privacy` _Privacy_ - The privacy setting of the asset. +- `cost` _Optional[Union[Dict, float]]_ - The cost associated with the asset. + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text, + description: Text, + supplier: Union[Dict, Text, Supplier, int] = Supplier.AIXPLAIN, + version: Text = "1.0", + license: Optional[License] = None, + privacy: Privacy = Privacy.PRIVATE, + cost: Optional[Union[Dict, float]] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/asset.py#L47) + +Initialize a new Asset instance. + +**Arguments**: + +- `id` _Text_ - Unique identifier of the asset. +- `name` _Text_ - Name of the asset. +- `description` _Text_ - Detailed description of the asset. +- `supplier` _Union[Dict, Text, Supplier, int], optional_ - Supplier of the asset. + Can be a Supplier enum, dictionary, text, or integer. Defaults to Supplier.AIXPLAIN. +- `version` _Text, optional_ - Version of the asset. Defaults to "1.0". +- `license` _Optional[License], optional_ - License associated with the asset. Defaults to None. +- `privacy` _Privacy, optional_ - Privacy setting of the asset. Defaults to Privacy.PRIVATE. +- `cost` _Optional[Union[Dict, float]], optional_ - Cost of the asset. Can be a dictionary + with pricing details or a float value. Defaults to None. + +#### to\_dict + +```python +def to_dict() -> dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/asset.py#L95) + +Convert the Asset instance to a dictionary representation. + +This method serializes all attributes of the Asset instance into a dictionary +format, which can be useful for data transmission or storage. + +**Returns**: + +- `dict` - A dictionary containing all attributes of the Asset instance. + Keys are attribute names and values are their corresponding values. + diff --git a/docs/api-reference/python/aixplain/modules/benchmark.md b/docs/api-reference/python/aixplain/modules/benchmark.md new file mode 100644 index 00000000..f35b253f --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/benchmark.md @@ -0,0 +1,118 @@ +--- +sidebar_label: benchmark +title: aixplain.modules.benchmark +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: October 25th 2022 +Description: + Benchmark Class + +### Benchmark Objects + +```python +class Benchmark(Asset) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/benchmark.py#L32) + +Benchmark is a powerful tool for benchmarking machine learning models and evaluating their performance on specific tasks. +It represents a collection of Models, Datasets and Metrics to run associated Benchmark Jobs. + +**Attributes**: + +- `id` _str_ - ID of the Benchmark. +- `name` _str_ - Name of the Benchmark. +- `model_list` _List[Model]_ - List of Models to be used for benchmarking. +- `dataset_list` _List[Dataset]_ - List of Datasets to be used for benchmarking. +- `metric_list` _List[Metric]_ - List of Metrics to be used for benchmarking. +- `job_list` _List[BenchmarkJob]_ - List of associated Benchmark Jobs. +- `additional_info` _dict_ - Any additional information to be saved with the Benchmark. + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text, + dataset_list: List[Dataset], + model_list: List[Model], + metric_list: List[Metric], + job_list: List[BenchmarkJob], + description: Text = "", + supplier: Text = "aiXplain", + version: Text = "1.0", + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/benchmark.py#L46) + +Create a Benchmark with the necessary information. + +**Arguments**: + +- `id` _Text_ - ID of the Benchmark. +- `name` _Text_ - Name of the Benchmark. +- `dataset_list` _List[Dataset]_ - List of Datasets to be used for benchmarking. +- `model_list` _List[Model]_ - List of Models to be used for benchmarking. +- `metric_list` _List[Metric]_ - List of Metrics to be used for benchmarking. +- `job_list` _List[BenchmarkJob]_ - List of associated Benchmark Jobs. +- `description` _Text, optional_ - Description of the Benchmark. Defaults to "". +- `supplier` _Text, optional_ - Author of the Benchmark. Defaults to "aiXplain". +- `version` _Text, optional_ - Benchmark version. Defaults to "1.0". +- `**additional_info` - Any additional Benchmark info to be saved. + +#### \_\_repr\_\_ + +```python +def __repr__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/benchmark.py#L83) + +Return a string representation of the Benchmark instance. + +**Returns**: + +- `str` - A string in the format "<Benchmark name>". + +#### start + +```python +def start() -> BenchmarkJob +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/benchmark.py#L91) + +Start a new benchmark job (run) for the current benchmark. + +This method initiates a new benchmark job using the configured models, +datasets, and metrics. It communicates with the backend API to create +and start the job. + +**Returns**: + +- `BenchmarkJob` - A new BenchmarkJob instance representing the started job. + Returns None if the job creation fails. + + +**Raises**: + +- `Exception` - If there's an error creating or starting the benchmark job. + The error is logged and None is returned. + diff --git a/docs/api-reference/python/aixplain/modules/benchmark_job.md b/docs/api-reference/python/aixplain/modules/benchmark_job.md new file mode 100644 index 00000000..9cdbefeb --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/benchmark_job.md @@ -0,0 +1,193 @@ +--- +sidebar_label: benchmark_job +title: aixplain.modules.benchmark_job +--- + +### BenchmarkJob Objects + +```python +class BenchmarkJob() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/benchmark_job.py#L12) + +Benchmark Job Represents a single run of an already created Benchmark. + +**Attributes**: + +- `id` _str_ - ID of the Benchmark Job. +- `status` _str_ - Status of the Benchmark Job. +- `benchmark_id` _str_ - ID of the associated parent Benchmark. +- `additional_info` _dict_ - Any additional information to be saved with the Benchmark Job. + +#### \_\_init\_\_ + +```python +def __init__(id: Text, status: Text, benchmark_id: Text, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/benchmark_job.py#L23) + +Create a Benchmark Job with the necessary information. Each Job is a run of a parent Benchmark + +**Arguments**: + +- `id` _Text_ - ID of the Benchmark Job +- `status` _Text_ - Status of the Benchmark Job +- `benchmark_id` _Text_ - ID of the associated parent Benchmark +- `**additional_info` - Any additional Benchmark Job info to be saved + +#### check\_status + +```python +def check_status() -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/benchmark_job.py#L81) + +Check the current status of the benchmark job. + +Fetches the latest status from the API and updates the local state. + +**Returns**: + +- `Text` - The current status of the benchmark job. + +#### download\_results\_as\_csv + +```python +def download_results_as_csv(save_path: Optional[Text] = None, + return_dataframe: bool = False) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/benchmark_job.py#L93) + +Get the results of the benchmark job in a CSV format. +The results can either be downloaded locally or returned in the form of pandas.DataFrame. + + +**Arguments**: + +- `save_path` _Text, optional_ - Path to save the CSV if return_dataframe is False. If None, a ranmdom path is generated. defaults to None. +- `return_dataframe` _bool_ - If True, the result is returned as pandas.DataFrame else saved as a CSV file. defaults to False. + + +**Returns**: + +- `str/pandas.DataFrame` - results as path of locally saved file if return_dataframe is False else as a pandas dataframe + +#### get\_scores + +```python +def get_scores( + return_simplified: bool = True, + return_as_dataframe: bool = True) -> Union[Dict, pd.DataFrame, list] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/benchmark_job.py#L149) + +Get the benchmark scores for all models. + +**Arguments**: + +- `return_simplified` _bool, optional_ - If True, returns a simplified version of scores. + Defaults to True. +- `return_as_dataframe` _bool, optional_ - If True and return_simplified is True, + returns results as a pandas DataFrame. Defaults to True. + + +**Returns**: + + Union[Dict, pd.DataFrame, list]: The benchmark scores in the requested format. + - If return_simplified=False: Returns a dictionary with detailed model scores + - If return_simplified=True and return_as_dataframe=True: Returns a pandas DataFrame + - If return_simplified=True and return_as_dataframe=False: Returns a list of dictionaries + + +**Raises**: + +- `Exception` - If there's an error fetching or processing the scores. + +#### get\_failuire\_rate + +```python +def get_failuire_rate( + return_as_dataframe: bool = True) -> Union[Dict, pd.DataFrame] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/benchmark_job.py#L197) + +Calculate the failure rate for each model in the benchmark. + +**Arguments**: + +- `return_as_dataframe` _bool, optional_ - If True, returns results as a pandas DataFrame. + Defaults to True. + + +**Returns**: + + Union[Dict, pd.DataFrame]: The failure rates for each model. + - If return_as_dataframe=True: Returns a DataFrame with 'Model' and 'Failure Rate' columns + - If return_as_dataframe=False: Returns a dictionary with model IDs as keys and failure rates as values + + +**Raises**: + +- `Exception` - If there's an error calculating the failure rates. + +#### get\_all\_explanations + +```python +def get_all_explanations() -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/benchmark_job.py#L236) + +Get all explanations for the benchmark results. + +**Returns**: + +- `Dict` - A dictionary containing both metric-dependent and metric-independent explanations. + The dictionary has two keys: + - 'metricInDependent': List of metric-independent explanations + - 'metricDependent': List of metric-dependent explanations + + +**Raises**: + +- `Exception` - If there's an error fetching the explanations. + +#### get\_localized\_explanations + +```python +def get_localized_explanations(metric_dependant: bool, + group_by_task: bool = False) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/benchmark_job.py#L261) + +Get localized explanations for the benchmark results. + +**Arguments**: + +- `metric_dependant` _bool_ - If True, returns metric-dependent explanations. + If False, returns metric-independent explanations. +- `group_by_task` _bool, optional_ - If True and metric_dependant is True, + groups explanations by task. Defaults to False. + + +**Returns**: + +- `Dict` - A dictionary containing the localized explanations. + The structure depends on the input parameters: + - If metric_dependant=False: Returns metric-independent explanations + - If metric_dependant=True and group_by_task=False: Returns explanations grouped by score ID + - If metric_dependant=True and group_by_task=True: Returns explanations grouped by task + + +**Raises**: + +- `Exception` - If there's an error fetching or processing the explanations. + diff --git a/docs/api-reference/python/aixplain/modules/content_interval.md b/docs/api-reference/python/aixplain/modules/content_interval.md new file mode 100644 index 00000000..1bc6edfa --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/content_interval.md @@ -0,0 +1,151 @@ +--- +sidebar_label: content_interval +title: aixplain.modules.content_interval +--- + +#### \_\_author\_\_ + +Copyright 2023 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: June 6th 2023 +Description: + Content Interval + +### ContentInterval Objects + +```python +@dataclass +class ContentInterval() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/content_interval.py#L29) + +Base class for representing intervals or segments within content. + +This class serves as the base for more specific content interval types +like text, audio, image, and video intervals. + +**Attributes**: + +- `content` _Text_ - The actual content within the interval. +- `content_id` _int_ - ID of the content interval. + +### TextContentInterval Objects + +```python +@dataclass +class TextContentInterval(ContentInterval) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/content_interval.py#L44) + +Class representing an interval or segment within text content. + +This class extends ContentInterval to handle text-specific intervals, +supporting both character-based and line-column-based positions. + +**Attributes**: + +- `content` _Text_ - The text content within the interval. +- `content_id` _int_ - ID of the content interval. +- `start` _Union[int, Tuple[int, int]]_ - Starting position of the interval. + Can be either a character offset (int) or a line-column tuple (int, int). +- `end` _Union[int, Tuple[int, int]]_ - Ending position of the interval. + Can be either a character offset (int) or a line-column tuple (int, int). + +### AudioContentInterval Objects + +```python +@dataclass +class AudioContentInterval(ContentInterval) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/content_interval.py#L63) + +Class representing an interval or segment within audio content. + +This class extends ContentInterval to handle audio-specific intervals +using timestamps. + +**Attributes**: + +- `content` _Text_ - The audio content within the interval. +- `content_id` _int_ - ID of the content interval. +- `start_time` _float_ - Starting timestamp of the interval in seconds. +- `end_time` _float_ - Ending timestamp of the interval in seconds. + +### ImageContentInterval Objects + +```python +@dataclass +class ImageContentInterval(ContentInterval) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/content_interval.py#L80) + +Class representing an interval or region within image content. + +This class extends ContentInterval to handle image-specific regions, +supporting both single points and polygons through coordinates. + +**Attributes**: + +- `content` _Text_ - The image content within the interval. +- `content_id` _int_ - ID of the content interval. +- `x` _Union[float, List[float]]_ - X-coordinate(s) of the region. + Single float for rectangular regions, list for polygon vertices. +- `y` _Union[float, List[float]]_ - Y-coordinate(s) of the region. + Single float for rectangular regions, list for polygon vertices. +- `width` _Optional[float]_ - Width of the region in pixels. Only used for + rectangular regions. Defaults to None. +- `height` _Optional[float]_ - Height of the region in pixels. Only used for + rectangular regions. Defaults to None. +- `rotation` _Optional[float]_ - Rotation angle of the region in degrees. + Defaults to None. + +### VideoContentInterval Objects + +```python +@dataclass +class VideoContentInterval(ContentInterval) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/content_interval.py#L108) + +Class representing an interval or region within video content. + +This class extends ContentInterval to handle video-specific intervals, +combining temporal information with optional spatial regions. + +**Attributes**: + +- `content` _Text_ - The video content within the interval. +- `content_id` _int_ - ID of the content interval. +- `start_time` _float_ - Starting timestamp of the interval in seconds. +- `end_time` _float_ - Ending timestamp of the interval in seconds. +- `x` _Optional[Union[float, List[float]]], optional_ - X-coordinate(s) of the region. + Single float for rectangular regions, list for polygon vertices. + Defaults to None. +- `y` _Optional[Union[float, List[float]]], optional_ - Y-coordinate(s) of the region. + Single float for rectangular regions, list for polygon vertices. + Defaults to None. +- `width` _Optional[float], optional_ - Width of the region in pixels. + Only used for rectangular regions. Defaults to None. +- `height` _Optional[float], optional_ - Height of the region in pixels. + Only used for rectangular regions. Defaults to None. +- `rotation` _Optional[float], optional_ - Rotation angle of the region in degrees. + Defaults to None. + diff --git a/docs/api-reference/python/aixplain/modules/corpus.md b/docs/api-reference/python/aixplain/modules/corpus.md new file mode 100644 index 00000000..485dc093 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/corpus.md @@ -0,0 +1,135 @@ +--- +sidebar_label: corpus +title: aixplain.modules.corpus +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: February 1st 2023 +Description: + Corpus Class + +### Corpus Objects + +```python +class Corpus(Asset) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/corpus.py#L37) + +A class representing a general-purpose collection of data in the aiXplain platform. + +This class extends Asset to provide functionality for managing corpora, which are +collections of data that can be processed and used to create task-specific datasets. +A corpus can contain various types of data and is used as a foundation for creating +specialized datasets. + +**Attributes**: + +- `id` _Text_ - ID of the corpus. +- `name` _Text_ - Name of the corpus. +- `description` _Text_ - Detailed description of the corpus. +- `data` _List[Data]_ - List of data objects that make up the corpus. +- `onboard_status` _OnboardStatus_ - Current onboarding status of the corpus. +- `functions` _List[Function]_ - AI functions the corpus is suitable for. +- `tags` _List[Text]_ - Descriptive tags for the corpus. +- `license` _Optional[License]_ - License associated with the corpus. +- `privacy` _Privacy_ - Privacy settings for the corpus. +- `supplier` _Text_ - The supplier/author of the corpus. +- `name`0 _Text_ - Version of the corpus. +- `name`1 _Optional[int]_ - Number of rows/items in the corpus. + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text, + description: Text, + data: List[Data], + onboard_status: OnboardStatus, + functions: List[Function] = [], + tags: List[Text] = [], + license: Optional[License] = None, + privacy: Privacy = Privacy.PRIVATE, + supplier: Text = "aiXplain", + version: Text = "1.0", + length: Optional[int] = None, + **kwargs) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/corpus.py#L60) + +Corpus Class. + +Description: +Corpus is general-purpose collection of data that can be processed and used to create task-specific datasets. + +**Arguments**: + +- `id` _Text_ - Corpus ID +- `name` _Text_ - Corpus Name +- `description` _Text_ - description of the corpus +- `data` _List[Data]_ - List of data which the corpus consists of +- `onboard_status` _OnboardStatus_ - onboard status +- `functions` _List[Function], optional_ - AI functions in which the corpus is suggested to be used to. Defaults to []. +- `tags` _List[Text], optional_ - description tags. Defaults to []. +- `license` _Optional[License], optional_ - Corpus license. Defaults to None. +- `privacy` _Privacy, optional_ - Corpus privacy info. Defaults to Privacy.PRIVATE. +- `supplier` _Text, optional_ - Corpus supplier. Defaults to "aiXplain". +- `name`0 _Text, optional_ - Corpus version. Defaults to "1.0". +- `name`1 _Optional[int], optional_ - Number of rows in the Corpus. Defaults to None. + +#### \_\_repr\_\_ + +```python +def __repr__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/corpus.py#L107) + +Return a string representation of the Corpus instance. + +**Returns**: + +- `str` - A string in the format "<Corpus: name>". + +#### delete + +```python +def delete() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/corpus.py#L115) + +Delete this corpus from the aiXplain platform. + +This method permanently removes the corpus from the platform. The operation +can only be performed by the corpus owner. + +**Returns**: + + None + + +**Raises**: + +- `Exception` - If the deletion fails, either because: + - The corpus doesn't exist + - The user is not the owner + - There's a network/server error + diff --git a/docs/api-reference/python/aixplain/modules/data.md b/docs/api-reference/python/aixplain/modules/data.md new file mode 100644 index 00000000..e25bf75c --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/data.md @@ -0,0 +1,101 @@ +--- +sidebar_label: data +title: aixplain.modules.data +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 20th 2023 +Description: + Data Class + +### Data Objects + +```python +class Data() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/data.py#L33) + +A class representing a collection of data samples of the same type and genre. + +This class provides functionality for managing data in the aiXplain platform, +supporting various data types, languages, and storage formats. It can handle +both structured (e.g., CSV) and unstructured data files. + +**Attributes**: + +- `id` _Text_ - ID of the data collection. +- `name` _Text_ - Name of the data collection. +- `dtype` _DataType_ - Type of data (e.g., text, audio, image). +- `privacy` _Privacy_ - Privacy settings for the data. +- `onboard_status` _OnboardStatus_ - Current onboarding status. +- `data_column` _Optional[Any]_ - Column identifier where data is stored in structured files. +- `start_column` _Optional[Any]_ - Column identifier for start indexes in structured files. +- `end_column` _Optional[Any]_ - Column identifier for end indexes in structured files. +- `files` _List[File]_ - List of files containing the data instances. +- `languages` _List[Language]_ - List of languages present in the data. +- `name`0 _DataSubtype_ - Subtype categorization of the data. +- `name`1 _Optional[int]_ - Number of samples/rows in the data collection. +- `name`2 _dict_ - Additional keyword arguments for extensibility. + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text, + dtype: DataType, + privacy: Privacy, + onboard_status: OnboardStatus, + data_column: Optional[Any] = None, + start_column: Optional[Any] = None, + end_column: Optional[Any] = None, + files: List[File] = [], + languages: List[Language] = [], + dsubtype: DataSubtype = DataSubtype.OTHER, + length: Optional[int] = None, + **kwargs) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/data.py#L56) + +Initialize a new Data instance. + +**Arguments**: + +- `id` _Text_ - ID of the data collection. +- `name` _Text_ - Name of the data collection. +- `dtype` _DataType_ - Type of data (e.g., text, audio, image). +- `privacy` _Privacy_ - Privacy settings for the data. +- `onboard_status` _OnboardStatus_ - Current onboarding status of the data. +- `data_column` _Optional[Any], optional_ - Column identifier where data is stored in + structured files (e.g., CSV). If None, defaults to the value of name. +- `start_column` _Optional[Any], optional_ - Column identifier where start indexes are + stored in structured files. Defaults to None. +- `end_column` _Optional[Any], optional_ - Column identifier where end indexes are + stored in structured files. Defaults to None. +- `files` _List[File], optional_ - List of files containing the data instances. + Defaults to empty list. +- `languages` _List[Language], optional_ - List of languages present in the data. + Can be provided as Language enums or language codes. Defaults to empty list. +- `name`0 _DataSubtype, optional_ - Subtype categorization of the data + (e.g., age, topic, race, split). Defaults to DataSubtype.OTHER. +- `name`1 _Optional[int], optional_ - Number of samples/rows in the data collection. + Defaults to None. +- `name`2 - Additional keyword arguments for extensibility. + diff --git a/docs/api-reference/python/aixplain/modules/dataset.md b/docs/api-reference/python/aixplain/modules/dataset.md new file mode 100644 index 00000000..f72d9e11 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/dataset.md @@ -0,0 +1,144 @@ +--- +sidebar_label: dataset +title: aixplain.modules.dataset +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: October 28th 2022 +Description: + Datasets Class + +### Dataset Objects + +```python +class Dataset(Asset) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/dataset.py#L38) + +Dataset is a collection of data intended to be used for a specific function. +Different from corpus, a dataset is a representative sample of a specific phenomenon to a specific AI task. +aiXplain also counts with an extensive collection of datasets for training, infer and benchmark various tasks like +Translation, Speech Recognition, Diacritization, Sentiment Analysis, and much more. + +**Attributes**: + +- `id` _Text_ - Dataset ID +- `name` _Text_ - Dataset Name +- `description` _Text_ - Dataset description +- `function` _Function_ - Function for which the dataset is intented to +- `source_data` _Dict[Any, Data]_ - List of input Data to the function +- `target_data` _Dict[Any, List[Data]]_ - List of Multi-reference Data which is expected to be outputted by the function +- `onboard_status` _OnboardStatus_ - onboard status +- `hypotheses` _Dict[Any, Data], optional_ - dataset's hypotheses, i.e. model outputs based on the source data. Defaults to \{}. +- `metadata` _Dict[Any, Data], optional_ - dataset's metadata. Defaults to \{}. +- `tags` _List[Text], optional_ - tags that describe the dataset. Defaults to []. +- `name`0 _Optional[License], optional_ - Dataset License. Defaults to None. +- `name`1 _Privacy, optional_ - Dataset Privacy. Defaults to Privacy.PRIVATE. +- `name`2 _Text, optional_ - Dataset Supplier. Defaults to "aiXplain". +- `name`3 _Text, optional_ - Dataset Version. Defaults to "1.0". + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text, + description: Text, + function: Function, + source_data: Dict[Any, Data], + target_data: Dict[Any, List[Data]], + onboard_status: OnboardStatus, + hypotheses: Dict[Any, Data] = {}, + metadata: Dict[Any, Data] = {}, + tags: List[Text] = [], + license: Optional[License] = None, + privacy: Privacy = Privacy.PRIVATE, + supplier: Text = "aiXplain", + version: Text = "1.0", + length: Optional[int] = None, + **kwargs) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/dataset.py#L61) + +Dataset Class. + +Description: +Dataset is a collection of data intended to be used for a specific function. +Different from corpus, a dataset is a representative sample of a specific phenomenon to a specific AI task. +aiXplain also counts with an extensive collection of datasets for training, infer and benchmark various tasks like +Translation, Speech Recognition, Diacritization, Sentiment Analysis, and much more. + +**Arguments**: + +- `id` _Text_ - Dataset ID +- `name` _Text_ - Dataset Name +- `description` _Text_ - Dataset description +- `function` _Function_ - Function for which the dataset is intented to +- `source_data` _Dict[Any, Data]_ - List of input Data to the function +- `target_data` _Dict[Any, List[Data]]_ - List of Multi-reference Data which is expected to be outputted by the function +- `onboard_status` _OnboardStatus_ - onboard status +- `hypotheses` _Dict[Any, Data], optional_ - dataset's hypotheses, i.e. model outputs based on the source data. Defaults to \{}. +- `metadata` _Dict[Any, Data], optional_ - dataset's metadata. Defaults to \{}. +- `tags` _List[Text], optional_ - tags that describe the dataset. Defaults to []. +- `name`0 _Optional[License], optional_ - Dataset License. Defaults to None. +- `name`1 _Privacy, optional_ - Dataset Privacy. Defaults to Privacy.PRIVATE. +- `name`2 _Text, optional_ - Dataset Supplier. Defaults to "aiXplain". +- `name`3 _Text, optional_ - Dataset Version. Defaults to "1.0". +- `name`4 _Optional[int], optional_ - Number of rows in the Dataset. Defaults to None. + +#### \_\_repr\_\_ + +```python +def __repr__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/dataset.py#L120) + +Return a string representation of the Dataset instance. + +**Returns**: + +- `str` - A string in the format "<Dataset: name>". + +#### delete + +```python +def delete() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/dataset.py#L128) + +Delete this dataset from the aiXplain platform. + +This method permanently removes the dataset from the platform. The operation +can only be performed by the dataset owner. + +**Returns**: + + None + + +**Raises**: + +- `Exception` - If the deletion fails, either because: + - The dataset doesn't exist + - The user is not the owner + - There's a network/server error + diff --git a/docs/api-reference/python/aixplain/modules/file.md b/docs/api-reference/python/aixplain/modules/file.md new file mode 100644 index 00000000..de816016 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/file.md @@ -0,0 +1,67 @@ +--- +sidebar_label: file +title: aixplain.modules.file +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 20th 2023 +Description: + File Class + +### File Objects + +```python +class File() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/file.py#L31) + +A class representing a file in the aiXplain platform. + +This class provides functionality for managing files, which are used to store +data samples in the platform. It supports various file types, compression formats, +and data splits. + +**Attributes**: + +- `path` _Union[Text, pathlib.Path]_ - File path +- `extension` _Union[Text, FileType]_ - File extension (e.g. CSV, TXT, etc.) +- `data_split` _Optional[DataSplit]_ - Data split of the file. +- `compression` _Optional[Text]_ - Compression extension (e.g., .gz). + +#### \_\_init\_\_ + +```python +def __init__(path: Union[Text, pathlib.Path], + extension: Union[Text, FileType], + data_split: Optional[DataSplit] = None, + compression: Optional[Text] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/file.py#L44) + +Initialize a new File instance. + +**Arguments**: + +- `path` _Union[Text, pathlib.Path]_ - File path +- `extension` _Union[Text, FileType]_ - File extension (e.g. CSV, TXT, etc.) +- `data_split` _Optional[DataSplit], optional_ - Data split of the file. Defaults to None. +- `compression` _Optional[Text], optional_ - Compression extension (e.g., .gz). Defaults to None. + diff --git a/docs/api-reference/python/aixplain/modules/finetune/cost.md b/docs/api-reference/python/aixplain/modules/finetune/cost.md new file mode 100644 index 00000000..7936c38b --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/finetune/cost.md @@ -0,0 +1,76 @@ +--- +sidebar_label: cost +title: aixplain.modules.finetune.cost +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: June 14th 2023 +Description: + FinetuneCost Class + +### FinetuneCost Objects + +```python +class FinetuneCost() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/finetune/cost.py#L27) + +A class representing the cost structure for a fine-tuning job. + +This class encapsulates the cost information for training, inference, and hosting +components of a fine-tuning job. It provides methods to convert the cost data +into a dictionary format for serialization. + +**Attributes**: + +- `training` _Dict_ - Dictionary containing training cost information. +- `inference` _Dict_ - Dictionary containing inference cost information. +- `hosting` _Dict_ - Dictionary containing hosting cost information. + +#### \_\_init\_\_ + +```python +def __init__(training: Dict, inference: Dict, hosting: Dict) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/finetune/cost.py#L40) + +Create a FinetuneCost object with training, inference, and hosting cost information. + +**Arguments**: + +- `training` _Dict_ - Dictionary containing training cost information. +- `inference` _Dict_ - Dictionary containing inference cost information. +- `hosting` _Dict_ - Dictionary containing hosting cost information. + +#### to\_dict + +```python +def to_dict() -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/finetune/cost.py#L57) + +Convert the FinetuneCost object to a dictionary. + +**Returns**: + +- `Dict` - A dictionary representation of the FinetuneCost object. + diff --git a/docs/api-reference/python/aixplain/modules/finetune/hyperparameters.md b/docs/api-reference/python/aixplain/modules/finetune/hyperparameters.md new file mode 100644 index 00000000..db91aa51 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/finetune/hyperparameters.md @@ -0,0 +1,76 @@ +--- +sidebar_label: hyperparameters +title: aixplain.modules.finetune.hyperparameters +--- + +### SchedulerType Objects + +```python +class SchedulerType(Text, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/finetune/hyperparameters.py#L7) + +Enum representing different learning rate schedulers. + +This enum defines the possible learning rate schedulers that can be used +in the fine-tuning process. Each scheduler is represented by a string constant. + +**Attributes**: + +- `LINEAR` _Text_ - Linear learning rate scheduler. +- `COSINE` _Text_ - Cosine learning rate scheduler. +- `COSINE_WITH_RESTARTS` _Text_ - Cosine with restarts learning rate scheduler. +- `POLYNOMIAL` _Text_ - Polynomial learning rate scheduler. +- `CONSTANT` _Text_ - Constant learning rate scheduler. +- `CONSTANT_WITH_WARMUP` _Text_ - Constant with warmup learning rate scheduler. +- `INVERSE_SQRT` _Text_ - Inverse square root learning rate scheduler. +- `REDUCE_ON_PLATEAU` _Text_ - Reduce learning rate on plateau learning rate scheduler. + +### Hyperparameters Objects + +```python +@dataclass_json + +@dataclass +class Hyperparameters(object) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/finetune/hyperparameters.py#L40) + +Configuration for the fine-tuning process. + +This class encapsulates the hyperparameters for training a model using a +fine-tuning approach. It includes settings for epochs, batch sizes, learning +rates, sequence lengths, and learning rate schedulers. + +**Attributes**: + +- `epochs` _int_ - Number of training epochs. +- `train_batch_size` _int_ - Batch size for training. +- `eval_batch_size` _int_ - Batch size for evaluation. +- `learning_rate` _float_ - Learning rate for training. +- `max_seq_length` _int_ - Maximum sequence length for model inputs. +- `warmup_ratio` _float_ - Warmup ratio for learning rate scheduler. +- `warmup_steps` _int_ - Number of warmup steps for learning rate scheduler. +- `lr_scheduler_type` _SchedulerType_ - Type of learning rate scheduler. + +#### \_\_post\_init\_\_ + +```python +def __post_init__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/finetune/hyperparameters.py#L66) + +Post-initialization validation for the hyperparameters. + +This method performs validation checks on the hyperparameters after +initialization. It ensures that the provided values are of the correct +types and within the allowed ranges. + +**Raises**: + +- `TypeError` - If the provided values are not of the correct types. +- `ValueError` - If the provided values are outside the allowed ranges. + diff --git a/docs/api-reference/python/aixplain/modules/finetune/init.md b/docs/api-reference/python/aixplain/modules/finetune/init.md new file mode 100644 index 00000000..96f73ae1 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/finetune/init.md @@ -0,0 +1,129 @@ +--- +sidebar_label: finetune +title: aixplain.modules.finetune +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: June 14th 2023 +Description: + FineTune Class + +### Finetune Objects + +```python +class Finetune(Asset) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/finetune/__init__.py#L37) + +A tool for fine-tuning machine learning models using custom datasets. + +This class provides functionality to customize pre-trained models for specific tasks +by fine-tuning them on user-provided datasets. It handles the configuration of +training parameters, data splitting, and job execution. + +**Attributes**: + +- `name` _Text_ - Name of the fine-tuning job. +- `dataset_list` _List[Dataset]_ - List of datasets to use for fine-tuning. +- `model` _Model_ - The base model to be fine-tuned. +- `cost` _FinetuneCost_ - Cost information for the fine-tuning job. +- `id` _Text_ - ID of the fine-tuning job. +- `description` _Text_ - Detailed description of the fine-tuning purpose. +- `supplier` _Text_ - Provider/creator of the fine-tuned model. +- `version` _Text_ - Version identifier of the fine-tuning job. +- `train_percentage` _float_ - Percentage of data to use for training. +- `dev_percentage` _float_ - Percentage of data to use for validation. +- `dataset_list`0 _Text_ - Template for formatting training examples, using + <<COLUMN_NAME>> to reference dataset columns. +- `dataset_list`1 _Hyperparameters_ - Configuration for the fine-tuning process. +- `dataset_list`2 _dict_ - Extra metadata for the fine-tuning job. +- `dataset_list`3 _str_ - URL endpoint for the backend API. +- `dataset_list`4 _str_ - Authentication key for API access. +- `dataset_list`5 _str_ - aiXplain-specific API key. + +#### \_\_init\_\_ + +```python +def __init__(name: Text, + dataset_list: List[Dataset], + model: Model, + cost: FinetuneCost, + id: Optional[Text] = "", + description: Optional[Text] = "", + supplier: Optional[Text] = "aiXplain", + version: Optional[Text] = "1.0", + train_percentage: Optional[float] = 100, + dev_percentage: Optional[float] = 0, + prompt_template: Optional[Text] = None, + hyperparameters: Optional[Hyperparameters] = None, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/finetune/__init__.py#L64) + +Initialize a new Finetune instance. + +**Arguments**: + +- `name` _Text_ - Name of the fine-tuning job. +- `dataset_list` _List[Dataset]_ - List of datasets to use for fine-tuning. +- `model` _Model_ - The base model to be fine-tuned. +- `cost` _FinetuneCost_ - Cost information for the fine-tuning job. +- `id` _Text, optional_ - ID of the job. Defaults to "". +- `description` _Text, optional_ - Detailed description of the fine-tuning + purpose. Defaults to "". +- `supplier` _Text, optional_ - Provider/creator of the fine-tuned model. + Defaults to "aiXplain". +- `version` _Text, optional_ - Version identifier. Defaults to "1.0". +- `train_percentage` _float, optional_ - Percentage of data to use for + training. Defaults to 100. +- `dev_percentage` _float, optional_ - Percentage of data to use for + validation. Defaults to 0. +- `dataset_list`0 _Text, optional_ - Template for formatting training + examples. Use <<COLUMN_NAME>> to reference dataset columns. + Defaults to None. +- `dataset_list`1 _Hyperparameters, optional_ - Configuration for the + fine-tuning process. Defaults to None. +- `dataset_list`2 - Extra metadata for the fine-tuning job. + +#### start + +```python +def start() -> Model +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/finetune/__init__.py#L117) + +Start the fine-tuning job on the backend. + +This method submits the fine-tuning configuration to the backend and initiates +the training process. It handles the creation of the training payload, +including dataset splits and hyperparameters. + +**Returns**: + +- `Model` - The model object representing the fine-tuning job. Returns None + if the job submission fails. + + +**Raises**: + +- `Exception` - If there are errors in the API request or response handling. + diff --git a/docs/api-reference/python/aixplain/modules/finetune/status.md b/docs/api-reference/python/aixplain/modules/finetune/status.md new file mode 100644 index 00000000..5981acb3 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/finetune/status.md @@ -0,0 +1,50 @@ +--- +sidebar_label: status +title: aixplain.modules.finetune.status +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: February 21st 2024 +Description: + FinetuneCost Class + +### FinetuneStatus Objects + +```python +@dataclass_json + +@dataclass +class FinetuneStatus(object) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/finetune/status.py#L32) + +Status information for a fine-tuning job. + +This class encapsulates the status of a fine-tuning job, including the overall +status of the job, the status of the model, and various training metrics. + +**Attributes**: + +- `status` _AssetStatus_ - Overall status of the fine-tuning job. +- `model_status` _AssetStatus_ - Status of the fine-tuned model. +- `epoch` _Optional[float]_ - Current training epoch. +- `training_loss` _Optional[float]_ - Training loss at the current epoch. +- `validation_loss` _Optional[float]_ - Validation loss at the current epoch. + diff --git a/docs/api-reference/python/aixplain/modules/init.md b/docs/api-reference/python/aixplain/modules/init.md new file mode 100644 index 00000000..1412d36c --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/init.md @@ -0,0 +1,25 @@ +--- +sidebar_label: modules +title: aixplain.modules +--- + +aiXplain SDK Library. +--- + +aiXplain SDK enables python programmers to add AI functions +to their software. + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + diff --git a/docs/api-reference/python/aixplain/modules/metadata.md b/docs/api-reference/python/aixplain/modules/metadata.md new file mode 100644 index 00000000..73cf376c --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/metadata.md @@ -0,0 +1,90 @@ +--- +sidebar_label: metadata +title: aixplain.modules.metadata +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain team +Date: March 20th 2023 +Description: + Meta-data Class + +### MetaData Objects + +```python +class MetaData() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/metadata.py#L33) + +A class representing metadata for data in the aiXplain platform. + +This class provides functionality for managing metadata, which is used to store +information about data in the platform. It supports various data types, languages, +and storage formats. + +**Attributes**: + +- `name` _Text_ - Name of the data. +- `dtype` _DataType_ - Type of data. +- `storage_type` _StorageType_ - Storage type of the data. +- `data_column` _Optional[Text]_ - Column index/name where the data is on a structured file. +- `start_column` _Optional[Text]_ - Column index/name where the start indexes is on a structured file. +- `end_column` _Optional[Text]_ - Column index/name where the end indexes is on a structured file. +- `privacy` _Optional[Privacy]_ - Privacy of data. +- `file_extension` _Optional[FileType]_ - File extension (e.g. CSV, TXT, etc.). +- `languages` _List[Language]_ - List of languages which the data consists of. +- `dsubtype` _DataSubtype_ - Data subtype (e.g., age, topic, race, split, etc.), used in datasets metadata. +- `dtype`0 _Optional[Text]_ - Data ID. +- `dtype`1 _dict_ - Additional keyword arguments for extensibility. + +#### \_\_init\_\_ + +```python +def __init__(name: Text, + dtype: DataType, + storage_type: StorageType, + data_column: Optional[Text] = None, + start_column: Optional[Text] = None, + end_column: Optional[Text] = None, + privacy: Optional[Privacy] = None, + file_extension: Optional[FileType] = None, + languages: List[Language] = [], + dsubtype: DataSubtype = DataSubtype.OTHER, + id: Optional[Text] = None, + **kwargs) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/metadata.py#L54) + +Initialize a new MetaData instance. + +**Arguments**: + +- `name` _Text_ - Data Name +- `dtype` _DataType_ - Data Type +- `storage_type` _StorageType_ - Data Storage (e.g. text, local file, web link) +- `data_column` _Optional[Text], optional_ - Column index/name where the data is on a structured file (e.g. CSV). Defaults to None. +- `start_column` _Optional[Text], optional_ - Column index/name where the start indexes is on a structured file (e.g. CSV). Defaults to None. +- `end_column` _Optional[Text], optional_ - Column index/name where the end indexes is on a structured file (e.g. CSV). Defaults to None. +- `privacy` _Optional[Privacy], optional_ - Privacy of data. Defaults to None. +- `file_extension` _Optional[FileType], optional_ - File extension (e.g. CSV, TXT, etc.). Defaults to None. +- `languages` _List[Language], optional_ - List of languages which the data consists of. Defaults to []. +- `dsubtype` _DataSubtype, optional_ - Data subtype (e.g., age, topic, race, split, etc.), used in datasets metadata. Defaults to Other. +- `dtype`0 _Optional[Text], optional_ - Data ID. Defaults to None. + diff --git a/docs/api-reference/python/aixplain/modules/metric.md b/docs/api-reference/python/aixplain/modules/metric.md new file mode 100644 index 00000000..272cdd4c --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/metric.md @@ -0,0 +1,156 @@ +--- +sidebar_label: metric +title: aixplain.modules.metric +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: October 25th 2022 +Description: + Metric Class + +### Metric Objects + +```python +class Metric(Asset) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/metric.py#L28) + +A class representing a metric for evaluating machine learning model outputs. + +This class extends Asset to provide functionality for computing evaluation metrics +on one or more pieces of data. Each metric is typically associated with a specific +machine learning task and can require different inputs (e.g., reference text for +translation metrics). + +**Attributes**: + +- `id` _Text_ - ID of the metric. +- `name` _Text_ - Name of the metric. +- `supplier` _Text_ - Author/provider of the metric. +- `is_reference_required` _bool_ - Whether the metric requires reference data. +- `is_source_required` _bool_ - Whether the metric requires source data. +- `cost` _float_ - Cost per metric computation. +- `function` _Text_ - The function identifier for this metric. +- `normalization_options` _list_ - List of available normalization options. +- `description` _Text_ - Description of the metric. +- `version` _Text_ - Version of the metric implementation. +- `name`0 _dict_ - Additional metric-specific information. + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text, + supplier: Text, + is_reference_required: bool, + is_source_required: bool, + cost: float, + function: Text, + normalization_options: list = [], + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/metric.py#L50) + +Initialize a new Metric instance. + +**Arguments**: + +- `id` _Text_ - ID of the metric. +- `name` _Text_ - Name of the metric. +- `supplier` _Text_ - Author/provider of the metric. +- `is_reference_required` _bool_ - Whether the metric requires reference data for computation. +- `is_source_required` _bool_ - Whether the metric requires source data for computation. +- `cost` _float_ - Cost per metric computation. +- `function` _Text_ - The function identifier for this metric. +- `normalization_options` _list, optional_ - List of available normalization options. + Defaults to empty list. +- `**additional_info` - Additional metric-specific information to be stored. + +#### \_\_repr\_\_ + +```python +def __repr__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/metric.py#L83) + +Return a string representation of the Metric instance. + +**Returns**: + +- `str` - A string in the format "<Metric name>". + +#### add\_normalization\_options + +```python +def add_normalization_options(normalization_options: List[str]) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/metric.py#L91) + +Add normalization options to be used during metric computation. + +This method appends new normalization options to the existing list of options. +These options can be used to normalize inputs or outputs during benchmarking. + +**Arguments**: + +- `normalization_options` _List[str]_ - List of normalization options to add. + Each option should be a valid normalization identifier. + +#### run + +```python +def run(hypothesis: Optional[Union[str, List[str]]] = None, + source: Optional[Union[str, List[str]]] = None, + reference: Optional[Union[str, List[str]]] = None) -> dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/metric.py#L103) + +Run the metric to calculate scores for the provided inputs. + +This method computes metric scores based on the provided hypothesis, and optionally +source and reference data. The inputs can be either single strings or lists of strings. + +**Arguments**: + +- `hypothesis` _Optional[Union[str, List[str]]], optional_ - The hypothesis/output to evaluate. + Can be a single string or a list of strings. Defaults to None. +- `source` _Optional[Union[str, List[str]]], optional_ - The source data for evaluation. + Only used if is_source_required is True. Can be a single string or a list + of strings. Defaults to None. +- `reference` _Optional[Union[str, List[str]]], optional_ - The reference data for evaluation. + Only used if is_reference_required is True. Can be a single string or a list + of strings. Defaults to None. + + +**Returns**: + +- `dict` - A dictionary containing the computed metric scores and any additional + computation metadata. + + +**Notes**: + + The method automatically handles conversion of single strings to lists and + proper formatting of references for multi-reference scenarios. + diff --git a/docs/api-reference/python/aixplain/modules/mixins.md b/docs/api-reference/python/aixplain/modules/mixins.md new file mode 100644 index 00000000..5ac4e4bf --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/mixins.md @@ -0,0 +1,60 @@ +--- +sidebar_label: mixins +title: aixplain.modules.mixins +--- + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: November 25th 2024 +Description: + Mixins for common functionality across different asset types + +### DeployableMixin Objects + +```python +class DeployableMixin(ABC, Generic[T]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/mixins.py#L28) + +A mixin that provides common deployment-related functionality for assets. + +This mixin provides methods for: +1. Filtering items that are not onboarded +2. Validating if an asset is ready to be deployed +3. Deploying an asset + +Classes that inherit from this mixin should: +1. Implement _validate_deployment_readiness to call the parent implementation with their specific asset type +2. Optionally override deploy() if they need special deployment handling + +#### deploy + +```python +def deploy() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/mixins.py#L58) + +Deploy the asset. + +This method validates that the asset is ready to be deployed and updates its status to ONBOARDED. +Classes that need special deployment handling should override this method. + +**Raises**: + +- `ValueError` - If the asset is not ready to be deployed + diff --git a/docs/api-reference/python/aixplain/modules/model/connection.md b/docs/api-reference/python/aixplain/modules/model/connection.md new file mode 100644 index 00000000..68f7de4b --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/model/connection.md @@ -0,0 +1,182 @@ +--- +sidebar_label: connection +title: aixplain.modules.model.connection +--- + +### ConnectAction Objects + +```python +class ConnectAction() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/connection.py#L7) + +A class representing an action that can be performed by a connection. + +This class defines the structure of a connection action with its name, description, +code, and input parameters. + +**Attributes**: + +- `name` _Text_ - The display name of the action. +- `description` _Text_ - A detailed description of what the action does. +- `code` _Optional[Text]_ - The internal code/identifier for the action. +- `inputs` _Optional[Dict]_ - The input parameters required by the action. + +#### \_\_init\_\_ + +```python +def __init__(name: Text, + description: Text, + code: Optional[Text] = None, + inputs: Optional[Dict] = None) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/connection.py#L25) + +Initialize a new ConnectAction instance. + +**Arguments**: + +- `name` _Text_ - The display name of the action. +- `description` _Text_ - A detailed description of what the action does. +- `code` _Optional[Text], optional_ - The internal code/identifier for the action. Defaults to None. +- `inputs` _Optional[Dict], optional_ - The input parameters required by the action. Defaults to None. + +#### \_\_repr\_\_ + +```python +def __repr__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/connection.py#L45) + +Return a string representation of the ConnectAction instance. + +**Returns**: + +- `str` - A string in the format "Action(code=<code>, name=<name>)". + +### ConnectionTool Objects + +```python +class ConnectionTool(Model) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/connection.py#L54) + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text, + description: Text = "", + api_key: Optional[Text] = None, + supplier: Union[Dict, Text, Supplier, int] = "aiXplain", + version: Optional[Text] = None, + function: Optional[Function] = None, + is_subscribed: bool = False, + cost: Optional[Dict] = None, + function_type: Optional[FunctionType] = FunctionType.CONNECTION, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/connection.py#L58) + +Initialize a new ConnectionTool instance. + +**Arguments**: + +- `id` _Text_ - ID of the Connection +- `name` _Text_ - Name of the Connection +- `description` _Text, optional_ - Description of the Connection. Defaults to "". +- `api_key` _Text, optional_ - API key of the Connection. Defaults to None. +- `supplier` _Union[Dict, Text, Supplier, int], optional_ - Supplier of the Connection. Defaults to "aiXplain". +- `version` _Text, optional_ - Version of the Connection. Defaults to "1.0". +- `function` _Function, optional_ - Function of the Connection. Defaults to None. +- `is_subscribed` _bool, optional_ - Is the user subscribed. Defaults to False. +- `cost` _Dict, optional_ - Cost of the Connection. Defaults to None. +- `function_type` _FunctionType, optional_ - Type of the Connection. Defaults to FunctionType.CONNECTION. +- `name`0 - Any additional Connection info to be saved + +#### get\_action\_inputs + +```python +def get_action_inputs(action: Union[ConnectAction, Text]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/connection.py#L131) + +Retrieve the input parameters required for a specific action. + +**Arguments**: + +- `action` _Union[ConnectAction, Text]_ - The action to get inputs for, either as a ConnectAction object + or as a string code. + + +**Returns**: + +- `Dict` - A dictionary containing the input parameters for the action. + + +**Raises**: + +- `Exception` - If the inputs cannot be retrieved from the server. + +#### run + +```python +def run(action: Union[ConnectAction, Text], inputs: Dict) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/connection.py#L165) + +Execute a specific action with the provided inputs. + +**Arguments**: + +- `action` _Union[ConnectAction, Text]_ - The action to execute, either as a ConnectAction object + or as a string code. +- `inputs` _Dict_ - The input parameters for the action. + + +**Returns**: + +- `Response` - The response from the server after executing the action. + +#### get\_parameters + +```python +def get_parameters() -> List[Dict] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/connection.py#L180) + +Get the parameters for all actions in the current action scope. + +**Returns**: + +- `List[Dict]` - A list of dictionaries containing the parameters for each action + in the action scope. Each dictionary contains the action's code, name, + description, and input parameters. + + +**Raises**: + +- `AssertionError` - If the action scope is not set or is empty. + +#### \_\_repr\_\_ + +```python +def __repr__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/connection.py#L205) + +Return a string representation of the ConnectionTool instance. + +**Returns**: + +- `str` - A string in the format "ConnectionTool(id=<id>, name=<name>)". + diff --git a/docs/api-reference/python/aixplain/modules/model/index_model.md b/docs/api-reference/python/aixplain/modules/model/index_model.md new file mode 100644 index 00000000..15b2f421 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/model/index_model.md @@ -0,0 +1,390 @@ +--- +sidebar_label: index_model +title: aixplain.modules.model.index_model +--- + +### IndexFilterOperator Objects + +```python +class IndexFilterOperator(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L16) + +Enumeration of operators available for filtering index records. + +This enum defines the comparison operators that can be used when creating +filters for searching and retrieving records from an index. + +**Attributes**: + +- `EQUALS` _str_ - Equality operator ("==") +- `NOT_EQUALS` _str_ - Inequality operator ("!=") +- `CONTAINS` _str_ - Membership test operator ("in") +- `NOT_CONTAINS` _str_ - Negative membership test operator ("not in") +- `GREATER_THAN` _str_ - Greater than operator (">") +- `LESS_THAN` _str_ - Less than operator ("<") +- `GREATER_THAN_OR_EQUALS` _str_ - Greater than or equal to operator (">=") +- `LESS_THAN_OR_EQUALS` _str_ - Less than or equal to operator ("<=") + +### IndexFilter Objects + +```python +class IndexFilter() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L42) + +A class representing a filter for querying index records. + +This class defines a filter that can be used to search or retrieve records from an index +based on specific field values and comparison operators. + +**Attributes**: + +- `field` _str_ - The name of the field to filter on. +- `value` _str_ - The value to compare against. +- `operator` _Union[IndexFilterOperator, str]_ - The comparison operator to use. + +#### \_\_init\_\_ + +```python +def __init__(field: str, value: str, operator: Union[IndexFilterOperator, + str]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L58) + +Initialize a new IndexFilter instance. + +**Arguments**: + +- `field` _str_ - The name of the field to filter on. +- `value` _str_ - The value to compare against. +- `operator` _Union[IndexFilterOperator, str]_ - The comparison operator to use. + +#### to\_dict + +```python +def to_dict() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L70) + +Convert the filter to a dictionary representation. + +**Returns**: + +- `dict` - A dictionary containing the filter's field, value, and operator. + The operator is converted to its string value if it's an IndexFilterOperator. + +### Splitter Objects + +```python +class Splitter() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L84) + +A class for configuring how documents should be split during indexing. + +This class provides options for splitting documents into smaller chunks before +they are indexed, which can be useful for large documents or for specific +search requirements. + +**Attributes**: + +- `split` _bool_ - Whether to split the documents or not. +- `split_by` _SplittingOptions_ - The method to use for splitting (e.g., by word, sentence). +- `split_length` _int_ - The length of each split chunk. +- `split_overlap` _int_ - The number of overlapping units between consecutive chunks. + +#### \_\_init\_\_ + +```python +def __init__(split: bool = False, + split_by: SplittingOptions = SplittingOptions.WORD, + split_length: int = 1, + split_overlap: int = 0) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L98) + +Initialize a new Splitter instance. + +**Arguments**: + +- `split` _bool, optional_ - Whether to split the documents. Defaults to False. +- `split_by` _SplittingOptions, optional_ - The method to use for splitting. + Defaults to SplittingOptions.WORD. +- `split_length` _int, optional_ - The length of each split chunk. Defaults to 1. +- `split_overlap` _int, optional_ - The number of overlapping units between + consecutive chunks. Defaults to 0. + +### IndexModel Objects + +```python +class IndexModel(Model) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L121) + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text, + description: Text = "", + api_key: Optional[Text] = None, + supplier: Union[Dict, Text, Supplier, int] = "aiXplain", + version: Optional[Text] = None, + function: Optional[Function] = None, + is_subscribed: bool = False, + cost: Optional[Dict] = None, + embedding_model: Union[EmbeddingModel, str] = None, + function_type: Optional[FunctionType] = FunctionType.SEARCH, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L122) + +Initialize a new IndexModel instance. + +**Arguments**: + +- `id` _Text_ - ID of the Index Model. +- `name` _Text_ - Name of the Index Model. +- `description` _Text, optional_ - Description of the Index Model. Defaults to "". +- `api_key` _Text, optional_ - API key of the Index Model. Defaults to None. +- `supplier` _Union[Dict, Text, Supplier, int], optional_ - Supplier of the Index Model. Defaults to "aiXplain". +- `version` _Text, optional_ - Version of the Index Model. Defaults to "1.0". +- `function` _Function, optional_ - Function of the Index Model. Must be Function.SEARCH. +- `is_subscribed` _bool, optional_ - Whether the user is subscribed. Defaults to False. +- `cost` _Dict, optional_ - Cost of the Index Model. Defaults to None. +- `embedding_model` _Union[EmbeddingModel, str], optional_ - Model used for embedding documents. Defaults to None. +- `name`0 _FunctionType, optional_ - Type of the function. Defaults to FunctionType.SEARCH. +- `name`1 - Any additional Index Model info to be saved. + + +**Raises**: + +- `name`2 - If function is not Function.SEARCH. + +#### to\_dict + +```python +def to_dict() -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L185) + +Convert the IndexModel instance to a dictionary representation. + +**Returns**: + +- `Dict` - A dictionary containing the model's attributes, including: + - All attributes from the parent Model class + - embedding_model: The model used for embedding documents + - embedding_size: The size of the embeddings produced + - collection_type: The type of collection derived from the version + +#### search + +```python +def search(query: str, + top_k: int = 10, + filters: List[IndexFilter] = []) -> ModelResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L201) + +Search for documents in the index + +**Arguments**: + +- `query` _str_ - Query to be searched +- `top_k` _int, optional_ - Number of results to be returned. Defaults to 10. +- `filters` _List[IndexFilter], optional_ - Filters to be applied. Defaults to []. + + +**Returns**: + +- `ModelResponse` - Response from the indexing service + + +**Example**: + + - index_model.search("Hello") + - index_model.search("", filters=[IndexFilter(field="category", value="animate", operator=IndexFilterOperator.EQUALS)]) + +#### upsert + +```python +def upsert(documents: List[Record], + splitter: Optional[Splitter] = None) -> ModelResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L234) + +Upsert documents into the index + +**Arguments**: + +- `documents` _List[Record]_ - List of documents to be upserted +- `splitter` _Splitter, optional_ - Splitter to be applied. Defaults to None. + + +**Returns**: + +- `ModelResponse` - Response from the indexing service + + +**Examples**: + + index_model.upsert([Record(value="Hello, world!", value_type="text", uri="", id="1", attributes=\{})]) + index_model.upsert([Record(value="Hello, world!", value_type="text", uri="", id="1", attributes=\{})], splitter=Splitter(split=True, split_by=SplittingOptions.WORD, split_length=1, split_overlap=0)) + Splitter in the above example is optional and can be used to split the documents into smaller chunks. + +#### count + +```python +def count() -> float +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L275) + +Get the total number of documents in the index. + +**Returns**: + +- `float` - The number of documents in the index. + + +**Raises**: + +- `Exception` - If the count operation fails. + + +**Example**: + + >>> index_model.count() + 42 + +#### get\_record + +```python +def get_record(record_id: Text) -> ModelResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L294) + +Get a document from the index. + +**Arguments**: + +- `record_id` _Text_ - ID of the document to retrieve. + + +**Returns**: + +- `ModelResponse` - Response containing the retrieved document data. + + +**Raises**: + +- `Exception` - If document retrieval fails. + + +**Example**: + + >>> index_model.get_record("123") + +#### delete\_record + +```python +def delete_record(record_id: Text) -> ModelResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L316) + +Delete a document from the index. + +**Arguments**: + +- `record_id` _Text_ - ID of the document to delete. + + +**Returns**: + +- `ModelResponse` - Response containing the deleted document data. + + +**Raises**: + +- `Exception` - If document deletion fails. + + +**Example**: + + >>> index_model.delete_record("123") + +#### retrieve\_records\_with\_filter + +```python +def retrieve_records_with_filter(filter: IndexFilter) -> ModelResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L338) + +Retrieve records from the index that match the given filter. + +**Arguments**: + +- `filter` _IndexFilter_ - The filter criteria to apply when retrieving records. + + +**Returns**: + +- `ModelResponse` - Response containing the retrieved records. + + +**Raises**: + +- `Exception` - If retrieval fails. + + +**Example**: + + >>> from aixplain.modules.model.index_model import IndexFilter, IndexFilterOperator + >>> my_filter = IndexFilter(field="category", value="world", operator=IndexFilterOperator.EQUALS) + >>> index_model.retrieve_records_with_filter(my_filter) + +#### delete\_records\_by\_date + +```python +def delete_records_by_date(date: float) -> ModelResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/index_model.py#L362) + +Delete records from the index that match the given date. + +**Arguments**: + +- `date` _float_ - The date (as a timestamp) to match records for deletion. + + +**Returns**: + +- `ModelResponse` - Response containing the result of the deletion operation. + + +**Raises**: + +- `Exception` - If deletion fails. + + +**Example**: + + >>> index_model.delete_records_by_date(1717708800) + diff --git a/docs/api-reference/python/aixplain/modules/model/init.md b/docs/api-reference/python/aixplain/modules/model/init.md new file mode 100644 index 00000000..5695084e --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/model/init.md @@ -0,0 +1,438 @@ +--- +sidebar_label: model +title: aixplain.modules.model +--- + +#### \_\_author\_\_ + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Duraikrishna Selvaraju, Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: September 1st 2022 +Description: + Model Class + +### Model Objects + +```python +class Model(Asset) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L41) + +A ready-to-use AI model that can be executed synchronously or asynchronously. + +This class represents a deployable AI model in the aiXplain platform. It provides +functionality for model execution, parameter management, and status tracking. +Models can be run with both synchronous and asynchronous APIs, and some models +support streaming responses. + +**Attributes**: + +- `id` _Text_ - ID of the model. +- `name` _Text_ - Name of the model. +- `description` _Text_ - Detailed description of the model's functionality. +- `api_key` _Text_ - Authentication key for API access. +- `url` _Text_ - Endpoint URL for model execution. +- `supplier` _Union[Dict, Text, Supplier, int]_ - Provider/creator of the model. +- `version` _Text_ - Version identifier of the model. +- `function` _Function_ - The AI function this model performs. +- `backend_url` _str_ - Base URL for the backend API. +- `cost` _Dict_ - Pricing information for model usage. +- `name`0 _ModelParameters_ - Parameters accepted by the model. +- `name`1 _Dict_ - Description of model outputs. +- `name`2 _ModelParameters_ - Configuration parameters for model behavior. +- `name`3 _bool_ - Whether the model supports streaming responses. +- `name`4 _FunctionType_ - Category of function (AI, UTILITY, etc.). +- `name`5 _bool_ - Whether the user has an active subscription. +- `name`6 _datetime_ - When the model was created. +- `name`7 _AssetStatus_ - Current status of the model. +- `name`8 _dict_ - Additional model metadata. + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text = "", + description: Text = "", + api_key: Text = config.TEAM_API_KEY, + supplier: Union[Dict, Text, Supplier, int] = "aiXplain", + version: Optional[Text] = None, + function: Optional[Function] = None, + is_subscribed: bool = False, + cost: Optional[Dict] = None, + created_at: Optional[datetime] = None, + input_params: Optional[Dict] = None, + output_params: Optional[Dict] = None, + model_params: Optional[Dict] = None, + supports_streaming: bool = False, + status: Optional[AssetStatus] = AssetStatus.ONBOARDED, + function_type: Optional[FunctionType] = FunctionType.AI, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L71) + +Initialize a new Model instance. + +**Arguments**: + +- `id` _Text_ - ID of the Model. +- `name` _Text, optional_ - Name of the Model. Defaults to "". +- `description` _Text, optional_ - Description of the Model. Defaults to "". +- `api_key` _Text, optional_ - Authentication key for API access. + Defaults to config.TEAM_API_KEY. +- `supplier` _Union[Dict, Text, Supplier, int], optional_ - Provider/creator + of the model. Defaults to "aiXplain". +- `version` _Text, optional_ - Version identifier of the model. Defaults to None. +- `function` _Function, optional_ - The AI function this model performs. + Defaults to None. +- `is_subscribed` _bool, optional_ - Whether the user has an active + subscription. Defaults to False. +- `cost` _Dict, optional_ - Pricing information for model usage. + Defaults to None. +- `created_at` _Optional[datetime], optional_ - When the model was created. + Defaults to None. +- `name`0 _Dict, optional_ - Parameters accepted by the model. + Defaults to None. +- `name`1 _Dict, optional_ - Description of model outputs. + Defaults to None. +- `name`2 _Dict, optional_ - Configuration parameters for model + behavior. Defaults to None. +- `name`3 _bool, optional_ - Whether the model supports streaming + responses. Defaults to False. +- `name`4 _AssetStatus, optional_ - Current status of the model. + Defaults to AssetStatus.ONBOARDED. +- `name`5 _FunctionType, optional_ - Category of function. + Defaults to FunctionType.AI. +- `name`6 - Additional model metadata. + +#### to\_dict + +```python +def to_dict() -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L144) + +Convert the model instance to a dictionary representation. + +**Returns**: + +- `Dict` - A dictionary containing the model's configuration with keys: + - id: Unique identifier + - name: Model name + - description: Model description + - supplier: Model provider + - additional_info: Extra metadata (excluding None/empty values) + - input_params: Input parameter configuration + - output_params: Output parameter configuration + - model_params: Model behavior parameters + - function: AI function type + - status: Current model status + +#### get\_parameters + +```python +def get_parameters() -> Optional[ModelParameters] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L174) + +Get the model's configuration parameters. + +**Returns**: + +- `Optional[ModelParameters]` - The model's parameter configuration if set, + None otherwise. + +#### \_\_repr\_\_ + +```python +def __repr__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L185) + +Return a string representation of the model. + +**Returns**: + +- `str` - A string in the format "Model: <name> by <supplier> (id=<id>)". + +#### sync\_poll + +```python +def sync_poll(poll_url: Text, + name: Text = "model_process", + wait_time: float = 0.5, + timeout: float = 300) -> ModelResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L196) + +Poll the platform until an asynchronous operation completes or times out. + +This method repeatedly checks the status of an asynchronous operation, +implementing exponential backoff for the polling interval. + +**Arguments**: + +- `poll_url` _Text_ - URL to poll for operation status. +- `name` _Text, optional_ - Identifier for the operation for logging. + Defaults to "model_process". +- `wait_time` _float, optional_ - Initial wait time in seconds between polls. + Will increase exponentially up to 60 seconds. Defaults to 0.5. +- `timeout` _float, optional_ - Maximum total time to poll in seconds. + Defaults to 300. + + +**Returns**: + +- `ModelResponse` - The final response from the operation. If polling times + out or fails, returns a failed response with appropriate error message. + + +**Notes**: + + The minimum wait time between polls is 0.2 seconds. The wait time + increases by 10% after each poll up to a maximum of 60 seconds. + +#### poll + +```python +def poll(poll_url: Text, name: Text = "model_process") -> ModelResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L262) + +Make a single poll request to check operation status. + +**Arguments**: + +- `poll_url` _Text_ - URL to poll for operation status. +- `name` _Text, optional_ - Identifier for the operation for logging. + Defaults to "model_process". + + +**Returns**: + +- `ModelResponse` - The current status of the operation. Contains completion + status, any results or errors, and usage statistics. + + +**Notes**: + + This is a low-level method used by sync_poll. Most users should use + sync_poll instead for complete operation handling. + +#### run\_stream + +```python +def run_stream(data: Union[Text, Dict], + parameters: Optional[Dict] = None) -> ModelResponseStreamer +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L312) + +Execute the model with streaming response. + +**Arguments**: + +- `data` _Union[Text, Dict]_ - The input data for the model. +- `parameters` _Optional[Dict], optional_ - Additional parameters for model + execution. Defaults to None. + + +**Returns**: + +- `ModelResponseStreamer` - A streamer object that yields response chunks. + + +**Raises**: + +- `AssertionError` - If the model doesn't support streaming. + +#### run + +```python +def run(data: Union[Text, Dict], + name: Text = "model_process", + timeout: float = 300, + parameters: Optional[Dict] = None, + wait_time: float = 0.5, + stream: bool = False) -> Union[ModelResponse, ModelResponseStreamer] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L338) + +Execute the model and wait for results. + +This method handles both synchronous and streaming execution modes. For +asynchronous operations, it polls until completion or timeout. + +**Arguments**: + +- `data` _Union[Text, Dict]_ - The input data for the model. +- `name` _Text, optional_ - Identifier for the operation for logging. + Defaults to "model_process". +- `timeout` _float, optional_ - Maximum time to wait for completion in seconds. + Defaults to 300. +- `parameters` _Dict, optional_ - Additional parameters for model execution. + Defaults to None. +- `wait_time` _float, optional_ - Initial wait time between polls in seconds. + Defaults to 0.5. +- `stream` _bool, optional_ - Whether to use streaming mode. Requires model + support. Defaults to False. + + +**Returns**: + + Union[ModelResponse, ModelResponseStreamer]: The model's response. For + streaming mode, returns a streamer object. For regular mode, + returns a response object with results or error information. + + +**Notes**: + + If the model execution becomes asynchronous, this method will poll + for completion using sync_poll with the specified timeout and wait_time. + +#### run\_async + +```python +def run_async(data: Union[Text, Dict], + name: Text = "model_process", + parameters: Optional[Dict] = None) -> ModelResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L408) + +Start asynchronous model execution. + +This method initiates model execution but doesn't wait for completion. +Use sync_poll to check the operation status later. + +**Arguments**: + +- `data` _Union[Text, Dict]_ - The input data for the model. +- `name` _Text, optional_ - Identifier for the operation for logging. + Defaults to "model_process". +- `parameters` _Dict, optional_ - Additional parameters for model execution. + Defaults to None. + + +**Returns**: + +- `ModelResponse` - Initial response containing: + - status: Current operation status + - url: URL for polling operation status + - error_message: Any immediate errors + - other response metadata + +#### check\_finetune\_status + +```python +def check_finetune_status(after_epoch: Optional[int] = None) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L447) + +Check the status of the FineTune model. + +**Arguments**: + +- `after_epoch` _Optional[int], optional_ - status after a given epoch. Defaults to None. + + +**Raises**: + +- `Exception` - If the 'TEAM_API_KEY' is not provided. + + +**Returns**: + +- `FinetuneStatus` - The status of the FineTune model. + +#### delete + +```python +def delete() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L514) + +Delete this model from the aiXplain platform. + +This method attempts to delete the model from the platform. It will fail +if the user doesn't have appropriate permissions. + +**Raises**: + +- `Exception` - If deletion fails or if the user doesn't have permission. + +#### add\_additional\_info\_for\_benchmark + +```python +def add_additional_info_for_benchmark(display_name: str, + configuration: Dict) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L538) + +Add benchmark-specific information to the model. + +This method updates the model's additional_info with benchmark-related +metadata. + +**Arguments**: + +- `display_name` _str_ - Name for display in benchmarks. +- `configuration` _Dict_ - Model configuration settings for benchmarking. + +#### from\_dict + +```python +@classmethod +def from_dict(cls, data: Dict) -> "Model" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/__init__.py#L552) + +Create a Model instance from a dictionary representation. + +**Arguments**: + +- `data` _Dict_ - Dictionary containing model configuration with keys: + - id: Model identifier + - name: Model name + - description: Model description + - api_key: API key for authentication + - supplier: Model provider information + - version: Model version + - function: AI function type + - is_subscribed: Subscription status + - cost: Pricing information + - created_at: Creation timestamp (ISO format) + - input_params: Input parameter configuration + - output_params: Output parameter configuration + - model_params: Model behavior parameters + - additional_info: Extra metadata + + +**Returns**: + +- `Model` - A new Model instance populated with the dictionary data. + diff --git a/docs/api-reference/python/aixplain/modules/model/integration.md b/docs/api-reference/python/aixplain/modules/model/integration.md new file mode 100644 index 00000000..9a3eefe6 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/model/integration.md @@ -0,0 +1,209 @@ +--- +sidebar_label: integration +title: aixplain.modules.model.integration +--- + +### AuthenticationSchema Objects + +```python +class AuthenticationSchema(Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/integration.py#L11) + +Enumeration of supported authentication schemes for integrations. + +This enum defines the various authentication methods that can be used +when connecting to external services through integrations. + +**Attributes**: + +- `BEARER_TOKEN` _str_ - Bearer token authentication scheme. +- `OAUTH1` _str_ - OAuth 1.0 authentication scheme. +- `OAUTH2` _str_ - OAuth 2.0 authentication scheme. +- `API_KEY` _str_ - API key authentication scheme. +- `BASIC` _str_ - Basic authentication scheme (username/password). +- `NO_AUTH` _str_ - No authentication required. + +### BaseAuthenticationParams Objects + +```python +class BaseAuthenticationParams(BaseModel) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/integration.py#L34) + +Base model for authentication parameters used in integrations. + +This class defines the common parameters that are used across different +authentication schemes when connecting to external services. + +**Attributes**: + +- `name` _Optional[Text]_ - Optional name for the connection. Defaults to None. +- `connector_id` _Optional[Text]_ - Optional ID of the connector. Defaults to None. + +#### build\_connector\_params + +```python +def build_connector_params(**kwargs) -> BaseAuthenticationParams +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/integration.py#L49) + +Build authentication parameters for a connector from keyword arguments. + +This function creates a BaseAuthenticationParams instance from the provided +keyword arguments, extracting the name and connector_id if present. + +**Arguments**: + +- `**kwargs` - Arbitrary keyword arguments. Supported keys: + - name (Optional[Text]): Name for the connection + - connector_id (Optional[Text]): ID of the connector + + +**Returns**: + +- `BaseAuthenticationParams` - An instance containing the extracted parameters. + + +**Example**: + + >>> params = build_connector_params(name="My Connection", connector_id="123") + >>> print(params.name) + 'My Connection' + +### Integration Objects + +```python +class Integration(Model) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/integration.py#L73) + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text, + description: Text = "", + api_key: Optional[Text] = None, + supplier: Union[Dict, Text, Supplier, int] = "aiXplain", + version: Optional[Text] = None, + function: Optional[Function] = None, + is_subscribed: bool = False, + cost: Optional[Dict] = None, + function_type: Optional[FunctionType] = FunctionType.INTEGRATION, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/integration.py#L74) + +Initialize a new Integration instance. + +**Arguments**: + +- `id` _Text_ - ID of the Integration. +- `name` _Text_ - Name of the Integration. +- `description` _Text, optional_ - Description of the Integration. Defaults to "". +- `api_key` _Text, optional_ - API key for the Integration. Defaults to None. +- `supplier` _Union[Dict, Text, Supplier, int], optional_ - Supplier of the Integration. Defaults to "aiXplain". +- `version` _Text, optional_ - Version of the Integration. Defaults to "1.0". +- `function` _Function, optional_ - Function of the Integration. Defaults to None. +- `is_subscribed` _bool, optional_ - Whether the user is subscribed. Defaults to False. +- `cost` _Dict, optional_ - Cost of the Integration. Defaults to None. +- `function_type` _FunctionType, optional_ - Type of the function. Must be FunctionType.INTEGRATION. + Defaults to FunctionType.INTEGRATION. +- `name`0 - Any additional Integration info to be saved. + + +**Raises**: + +- `name`1 - If function_type is not FunctionType.INTEGRATION. + +#### connect + +```python +def connect(authentication_schema: AuthenticationSchema, + args: Optional[BaseAuthenticationParams] = None, + data: Optional[Dict] = None, + **kwargs) -> ModelResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/integration.py#L127) + +Connect to the integration using the specified authentication scheme. + +This method establishes a connection to the integration service using the provided +authentication method and credentials. The required parameters vary depending on +the authentication scheme being used. + +**Arguments**: + +- `authentication_schema` _AuthenticationSchema_ - The authentication scheme to use + (e.g., BEARER_TOKEN, OAUTH1, OAUTH2, API_KEY, BASIC, NO_AUTH). +- `args` _Optional[BaseAuthenticationParams], optional_ - Common connection parameters. + If not provided, will be built from kwargs. Defaults to None. +- `data` _Optional[Dict], optional_ - Authentication-specific parameters required by + the chosen authentication scheme. Defaults to None. +- `**kwargs` - Additional keyword arguments used to build BaseAuthenticationParams + if args is not provided. Supported keys: + - name (str): Name for the connection + - connector_id (str): ID of the connector + + +**Returns**: + +- `ModelResponse` - A response object containing: + - data (Dict): Contains connection details including: + - id (str): Connection ID (can be used with ModelFactory.get(id)) + - redirectURL (str, optional): URL to complete OAuth authentication + (only for OAuth1/OAuth2) + + +**Raises**: + +- `ValueError` - If the authentication schema is not supported by this integration + or if required parameters are missing from the data dictionary. + + +**Examples**: + + Using Bearer Token authentication: + >>> integration.connect( + ... AuthenticationSchema.BEARER_TOKEN, + ... data=\{"token": "1234567890"}, + ... name="My Connection" + ... ) + + Using OAuth2 authentication: + >>> response = integration.connect( + ... AuthenticationSchema.OAUTH2, + ... name="My Connection" + ... ) + >>> # For OAuth2, you'll need to visit the redirectURL to complete auth + >>> print(response.data.get("redirectURL")) + + Using API Key authentication: + >>> integration.connect( + ... AuthenticationSchema.API_KEY, + ... data=\{"api_key": "your-api-key"}, + ... name="My Connection" + ... ) + +#### \_\_repr\_\_ + +```python +def __repr__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/integration.py#L242) + +Return a string representation of the Integration instance. + +**Returns**: + +- `str` - A string in the format "Integration: <name> by <supplier> (id=<id>)". + If supplier is a dictionary, uses supplier['name'], otherwise uses supplier directly. + diff --git a/docs/api-reference/python/aixplain/modules/model/llm_model.md b/docs/api-reference/python/aixplain/modules/model/llm_model.md new file mode 100644 index 00000000..8730ae20 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/model/llm_model.md @@ -0,0 +1,207 @@ +--- +sidebar_label: llm_model +title: aixplain.modules.model.llm_model +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: June 4th 2024 +Description: + Large Language Model Class + +### LLM Objects + +```python +class LLM(Model) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/llm_model.py#L36) + +Ready-to-use LLM model. This model can be run in both synchronous and asynchronous manner. + +**Attributes**: + +- `id` _Text_ - ID of the Model +- `name` _Text_ - Name of the Model +- `description` _Text, optional_ - description of the model. Defaults to "". +- `api_key` _Text, optional_ - API key of the Model. Defaults to None. +- `url` _Text, optional_ - endpoint of the model. Defaults to config.MODELS_RUN_URL. +- `supplier` _Union[Dict, Text, Supplier, int], optional_ - supplier of the asset. Defaults to "aiXplain". +- `version` _Text, optional_ - version of the model. Defaults to "1.0". +- `function` _Text, optional_ - model AI function. Defaults to None. +- `url` _str_ - URL to run the model. +- `backend_url` _str_ - URL of the backend. +- `name`0 _Dict, optional_ - model price. Defaults to None. +- `name`1 _FunctionType, optional_ - type of the function. Defaults to FunctionType.AI. +- `name`2 - Any additional Model info to be saved + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text, + description: Text = "", + api_key: Optional[Text] = None, + supplier: Union[Dict, Text, Supplier, int] = "aiXplain", + version: Optional[Text] = None, + function: Optional[Function] = None, + is_subscribed: bool = False, + cost: Optional[Dict] = None, + temperature: float = 0.001, + function_type: Optional[FunctionType] = FunctionType.AI, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/llm_model.py#L55) + +Initialize a new LLM instance. + +**Arguments**: + +- `id` _Text_ - ID of the LLM model. +- `name` _Text_ - Name of the LLM model. +- `description` _Text, optional_ - Description of the model. Defaults to "". +- `api_key` _Text, optional_ - API key for the model. Defaults to None. +- `supplier` _Union[Dict, Text, Supplier, int], optional_ - Supplier of the model. Defaults to "aiXplain". +- `version` _Text, optional_ - Version of the model. Defaults to "1.0". +- `function` _Function, optional_ - Model's AI function. Must be Function.TEXT_GENERATION. +- `is_subscribed` _bool, optional_ - Whether the user is subscribed. Defaults to False. +- `cost` _Dict, optional_ - Cost of the model. Defaults to None. +- `temperature` _float, optional_ - Default temperature for text generation. Defaults to 0.001. +- `name`0 _FunctionType, optional_ - Type of the function. Defaults to FunctionType.AI. +- `name`1 - Any additional model info to be saved. + + +**Raises**: + +- `name`2 - If function is not Function.TEXT_GENERATION. + +#### run + +```python +def run(data: Text, + context: Optional[Text] = None, + prompt: Optional[Text] = None, + history: Optional[List[Dict]] = None, + temperature: Optional[float] = None, + max_tokens: int = 128, + top_p: float = 1.0, + name: Text = "model_process", + timeout: float = 300, + parameters: Optional[Dict] = None, + wait_time: float = 0.5, + stream: bool = False) -> Union[ModelResponse, ModelResponseStreamer] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/llm_model.py#L107) + +Run the LLM model synchronously to generate text. + +This method runs the LLM model to generate text based on the provided input. +It supports both single-turn and conversational interactions, with options +for streaming responses. + +**Arguments**: + +- `data` _Text_ - The input text or last user utterance for text generation. +- `context` _Optional[Text], optional_ - System message or context for the model. + Defaults to None. +- `prompt` _Optional[Text], optional_ - Prompt template or prefix to prepend to + the input. Defaults to None. +- `history` _Optional[List[Dict]], optional_ - Conversation history in OpenAI format + (e.g., [\{"role": "assistant", "content": "Hello!"}, ...]). Defaults to None. +- `temperature` _Optional[float], optional_ - Sampling temperature for text generation. + Higher values make output more random. If None, uses the model's default. + Defaults to None. +- `max_tokens` _int, optional_ - Maximum number of tokens to generate. + Defaults to 128. +- `top_p` _float, optional_ - Nucleus sampling parameter. Only tokens with cumulative + probability < top_p are considered. Defaults to 1.0. +- `name` _Text, optional_ - Identifier for this model run. Useful for logging. + Defaults to "model_process". +- `timeout` _float, optional_ - Maximum time in seconds to wait for completion. + Defaults to 300. +- `parameters` _Optional[Dict], optional_ - Additional model-specific parameters. + Defaults to None. +- `context`0 _float, optional_ - Time in seconds between polling attempts. + Defaults to 0.5. +- `context`1 _bool, optional_ - Whether to stream the model's output tokens. + Defaults to False. + + +**Returns**: + + Union[ModelResponse, ModelResponseStreamer]: If stream=False, returns a ModelResponse + containing the complete generated text and metadata. If stream=True, returns + a ModelResponseStreamer that yields tokens as they're generated. + +#### run\_async + +```python +def run_async(data: Text, + context: Optional[Text] = None, + prompt: Optional[Text] = None, + history: Optional[List[Dict]] = None, + temperature: Optional[float] = None, + max_tokens: int = 128, + top_p: float = 1.0, + name: Text = "model_process", + parameters: Optional[Dict] = None) -> ModelResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/llm_model.py#L205) + +Run the LLM model asynchronously to generate text. + +This method starts an asynchronous text generation task and returns immediately +with a response containing a polling URL. The actual result can be retrieved +later using the polling URL. + +**Arguments**: + +- `data` _Text_ - The input text or last user utterance for text generation. +- `context` _Optional[Text], optional_ - System message or context for the model. + Defaults to None. +- `prompt` _Optional[Text], optional_ - Prompt template or prefix to prepend to + the input. Defaults to None. +- `history` _Optional[List[Dict]], optional_ - Conversation history in OpenAI format + (e.g., [\{"role": "assistant", "content": "Hello!"}, ...]). Defaults to None. +- `temperature` _Optional[float], optional_ - Sampling temperature for text generation. + Higher values make output more random. If None, uses the model's default. + Defaults to None. +- `max_tokens` _int, optional_ - Maximum number of tokens to generate. + Defaults to 128. +- `top_p` _float, optional_ - Nucleus sampling parameter. Only tokens with cumulative + probability < top_p are considered. Defaults to 1.0. +- `name` _Text, optional_ - Identifier for this model run. Useful for logging. + Defaults to "model_process". +- `parameters` _Optional[Dict], optional_ - Additional model-specific parameters. + Defaults to None. + + +**Returns**: + +- `ModelResponse` - A response object containing: + - status (ResponseStatus): Status of the request (e.g., IN_PROGRESS) + - url (str): URL to poll for the final result + - data (str): Empty string (result not available yet) + - details (Dict): Additional response details + - completed (bool): False (task not completed yet) + - error_message (str): Error message if request failed + Other fields may be present depending on the response. + diff --git a/docs/api-reference/python/aixplain/modules/model/mcp_connection.md b/docs/api-reference/python/aixplain/modules/model/mcp_connection.md new file mode 100644 index 00000000..385c6029 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/model/mcp_connection.md @@ -0,0 +1,140 @@ +--- +sidebar_label: mcp_connection +title: aixplain.modules.model.mcp_connection +--- + +### ConnectAction Objects + +```python +class ConnectAction() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/mcp_connection.py#L7) + +A class representing an action that can be performed by an MCP connection. + +This class defines the structure of a connection action with its name, description, +code, and input parameters. + +**Attributes**: + +- `name` _Text_ - The display name of the action. +- `description` _Text_ - A detailed description of what the action does. +- `code` _Optional[Text]_ - The internal code/identifier for the action. Defaults to None. +- `inputs` _Optional[Dict]_ - The input parameters required by the action. Defaults to None. + +#### \_\_init\_\_ + +```python +def __init__(name: Text, + description: Text, + code: Optional[Text] = None, + inputs: Optional[Dict] = None) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/mcp_connection.py#L25) + +Initialize a new ConnectAction instance. + +**Arguments**: + +- `name` _Text_ - The display name of the action. +- `description` _Text_ - A detailed description of what the action does. +- `code` _Optional[Text], optional_ - The internal code/identifier for the action. + Defaults to None. +- `inputs` _Optional[Dict], optional_ - The input parameters required by the action. + Defaults to None. + +#### \_\_repr\_\_ + +```python +def __repr__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/mcp_connection.py#L47) + +Return a string representation of the ConnectAction instance. + +**Returns**: + +- `str` - A string in the format "Action(code=<code>, name=<name>)". + +### MCPConnection Objects + +```python +class MCPConnection(ConnectionTool) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/mcp_connection.py#L56) + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text, + description: Text = "", + api_key: Optional[Text] = None, + supplier: Union[Dict, Text, Supplier, int] = "aiXplain", + version: Optional[Text] = None, + function: Optional[Function] = None, + is_subscribed: bool = False, + cost: Optional[Dict] = None, + function_type: Optional[FunctionType] = FunctionType.CONNECTION, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/mcp_connection.py#L60) + +Initialize a new MCPConnection instance. + +**Arguments**: + +- `id` _Text_ - ID of the MCP Connection. +- `name` _Text_ - Name of the MCP Connection. +- `description` _Text, optional_ - Description of the Connection. Defaults to "". +- `api_key` _Text, optional_ - API key for the Connection. Defaults to None. +- `supplier` _Union[Dict, Text, Supplier, int], optional_ - Supplier of the Connection. + Defaults to "aiXplain". +- `version` _Text, optional_ - Version of the Connection. Defaults to "1.0". +- `function` _Function, optional_ - Function of the Connection. Defaults to None. +- `is_subscribed` _bool, optional_ - Whether the user is subscribed. Defaults to False. +- `cost` _Dict, optional_ - Cost of the Connection. Defaults to None. +- `function_type` _FunctionType, optional_ - Type of the function. Must be + FunctionType.MCP_CONNECTION. Defaults to FunctionType.CONNECTION. +- `name`0 - Any additional Connection info to be saved. + + +**Raises**: + +- `name`1 - If function_type is not FunctionType.MCP_CONNECTION. + +#### get\_action\_inputs + +```python +def get_action_inputs(action: Union[ConnectAction, Text]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/mcp_connection.py#L136) + +Retrieve the input parameters required for a specific tool. + +This method fetches the input parameters that are required to use a specific +tool. If the action object already has its inputs cached, returns those +instead of making a server request. + +**Arguments**: + +- `action` _Union[ConnectAction, Text]_ - The tool to get inputs for, either as + a ConnectAction object or as a string code. + + +**Returns**: + +- `Dict` - A dictionary mapping input parameter codes to their specifications. + + +**Raises**: + +- `Exception` - If the inputs cannot be retrieved from the server or if the + response cannot be parsed. + diff --git a/docs/api-reference/python/aixplain/modules/model/model_parameters.md b/docs/api-reference/python/aixplain/modules/model/model_parameters.md new file mode 100644 index 00000000..2cc1840e --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/model/model_parameters.md @@ -0,0 +1,27 @@ +--- +sidebar_label: model_parameters +title: aixplain.modules.model.model_parameters +--- + +### ModelParameters Objects + +```python +class ModelParameters(BaseParameters) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/model_parameters.py#L5) + +#### \_\_init\_\_ + +```python +def __init__(input_params: Dict[str, Dict[str, Any]]) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/model_parameters.py#L6) + +Initialize ModelParameters with input parameters dictionary. + +**Arguments**: + +- `input_params` _Dict[str, Dict[str, Any]]_ - Dictionary containing parameter configurations + diff --git a/docs/api-reference/python/aixplain/modules/model/model_response_streamer.md b/docs/api-reference/python/aixplain/modules/model/model_response_streamer.md new file mode 100644 index 00000000..660c7c9d --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/model/model_response_streamer.md @@ -0,0 +1,61 @@ +--- +sidebar_label: model_response_streamer +title: aixplain.modules.model.model_response_streamer +--- + +### ModelResponseStreamer Objects + +```python +class ModelResponseStreamer() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/model_response_streamer.py#L7) + +A class representing a streamer for model responses. + +This class provides an iterator interface for streaming model responses. +It handles the conversion of JSON-like strings into ModelResponse objects +and manages the response status. + +#### \_\_init\_\_ + +```python +def __init__(iterator: Iterator) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/model_response_streamer.py#L15) + +Initialize a new ModelResponseStreamer instance. + +**Arguments**: + +- `iterator` _Iterator_ - An iterator that yields JSON-like strings. + +#### \_\_next\_\_ + +```python +def __next__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/model_response_streamer.py#L24) + +Return the next chunk of the response. + +**Returns**: + +- `ModelResponse` - A ModelResponse object containing the next chunk of the response. + +#### \_\_iter\_\_ + +```python +def __iter__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/model_response_streamer.py#L41) + +Return the iterator for the ModelResponseStreamer. + +**Returns**: + +- `Iterator` - The iterator for the ModelResponseStreamer. + diff --git a/docs/api-reference/python/aixplain/modules/model/record.md b/docs/api-reference/python/aixplain/modules/model/record.md new file mode 100644 index 00000000..8b249dc1 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/model/record.md @@ -0,0 +1,68 @@ +--- +sidebar_label: record +title: aixplain.modules.model.record +--- + +### Record Objects + +```python +class Record() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/record.py#L6) + +A class representing a record in an index. + +This class defines the structure of a record with its value, type, ID, URI, +and attributes. + +#### \_\_init\_\_ + +```python +def __init__(value: str = "", + value_type: DataType = DataType.TEXT, + id: Optional[str] = None, + uri: str = "", + attributes: dict = {}) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/record.py#L12) + +Initialize a new Record instance. + +**Arguments**: + +- `value` _str_ - The value of the record. +- `value_type` _DataType_ - The type of the value. +- `id` _Optional[str]_ - The ID of the record. Defaults to a random UUID. +- `uri` _str_ - The URI of the record. +- `attributes` _dict_ - The attributes of the record. + +#### to\_dict + +```python +def to_dict() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/record.py#L35) + +Convert the record to a dictionary. + +**Returns**: + +- `dict` - A dictionary containing the record's value, type, ID, URI, and attributes. + +#### validate + +```python +def validate() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/record.py#L49) + +Validate the record. + +**Raises**: + +- `AssertionError` - If the value type is invalid or if the URI is required for image records. + diff --git a/docs/api-reference/python/aixplain/modules/model/response.md b/docs/api-reference/python/aixplain/modules/model/response.md new file mode 100644 index 00000000..27f19d96 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/model/response.md @@ -0,0 +1,164 @@ +--- +sidebar_label: response +title: aixplain.modules.model.response +--- + +### ModelResponse Objects + +```python +class ModelResponse() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/response.py#L6) + +ModelResponse class to store the response of the model run. + +This class provides a structured way to store and manage the response from model runs. +It includes fields for status, data, details, completion status, error messages, +usage information, and additional metadata. + +#### \_\_init\_\_ + +```python +def __init__(status: ResponseStatus, + data: Text = "", + details: Optional[Union[Dict, List]] = {}, + completed: bool = False, + error_message: Text = "", + used_credits: float = 0.0, + run_time: float = 0.0, + usage: Optional[Dict] = None, + url: Optional[Text] = None, + error_code: Optional[ErrorCode] = None, + **kwargs) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/response.py#L14) + +Initialize a new ModelResponse instance. + +**Arguments**: + +- `status` _ResponseStatus_ - The status of the response. +- `data` _Text_ - The data returned by the model. +- `details` _Optional[Union[Dict, List]]_ - Additional details about the response. +- `completed` _bool_ - Whether the response is complete. +- `error_message` _Text_ - The error message if the response is not successful. +- `used_credits` _float_ - The amount of credits used for the response. +- `run_time` _float_ - The time taken to generate the response. +- `usage` _Optional[Dict]_ - Usage information about the response. +- `url` _Optional[Text]_ - The URL of the response. +- `error_code` _Optional[ErrorCode]_ - The error code if the response is not successful. +- `data`0 - Additional keyword arguments. + +#### \_\_getitem\_\_ + +```python +def __getitem__(key: Text) -> Any +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/response.py#L59) + +Get an item from the ModelResponse. + +**Arguments**: + +- `key` _Text_ - The key to get the value for. + + +**Returns**: + +- `Any` - The value associated with the key. + + +**Raises**: + +- `KeyError` - If the key is not found in the ModelResponse. + +#### get + +```python +def get(key: Text, default: Optional[Any] = None) -> Any +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/response.py#L81) + +Get an item from the ModelResponse with a default value. + +**Arguments**: + +- `key` _Text_ - The key to get the value for. +- `default` _Optional[Any]_ - The default value to return if the key is not found. + + +**Returns**: + +- `Any` - The value associated with the key or the default value if the key is not found. + +#### \_\_setitem\_\_ + +```python +def __setitem__(key: Text, value: Any) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/response.py#L96) + +Set an item in the ModelResponse. + +**Arguments**: + +- `key` _Text_ - The key to set the value for. +- `value` _Any_ - The value to set. + + +**Raises**: + +- `KeyError` - If the key is not found in the ModelResponse. + +#### \_\_repr\_\_ + +```python +def __repr__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/response.py#L117) + +Return a string representation of the ModelResponse. + +**Returns**: + +- `str` - A string representation of the ModelResponse. + +#### \_\_contains\_\_ + +```python +def __contains__(key: Text) -> bool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/response.py#L148) + +Check if a key is in the ModelResponse. + +**Arguments**: + +- `key` _Text_ - The key to check for. + + +**Returns**: + +- `bool` - True if the key is in the ModelResponse, False otherwise. + +#### to\_dict + +```python +def to_dict() -> Dict[Text, Any] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/response.py#L163) + +Convert the ModelResponse to a dictionary. + +**Returns**: + + Dict[Text, Any]: A dictionary representation of the ModelResponse. + diff --git a/docs/api-reference/python/aixplain/modules/model/utility_model.md b/docs/api-reference/python/aixplain/modules/model/utility_model.md new file mode 100644 index 00000000..f896b67d --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/model/utility_model.md @@ -0,0 +1,313 @@ +--- +sidebar_label: utility_model +title: aixplain.modules.model.utility_model +--- + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: November 25th 2024 +Description: + Utility Model Class + +### BaseUtilityModelParams Objects + +```python +class BaseUtilityModelParams(BaseModel) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utility_model.py#L36) + +Base model for utility model parameters. + +This class defines the basic parameters required to create or update a utility model. + +**Attributes**: + +- `name` _Text_ - The name of the utility model. +- `code` _Union[Text, Callable]_ - The implementation code, either as a string or + a callable function. +- `description` _Optional[Text]_ - A description of what the utility model does. + Defaults to None. + +### UtilityModelInput Objects + +```python +@dataclass +class UtilityModelInput() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utility_model.py#L55) + +A class representing an input parameter for a utility model. + +This class defines the structure and validation rules for input parameters +that can be used with utility models. + +**Attributes**: + +- `name` _Text_ - The name of the input parameter. +- `description` _Text_ - A description of what this input parameter represents. +- `type` _DataType_ - The data type of the input parameter. Must be one of: + TEXT, BOOLEAN, or NUMBER. Defaults to DataType.TEXT. + +#### validate + +```python +def validate() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utility_model.py#L72) + +Validate that the input parameter has a supported data type. + +**Raises**: + +- `ValueError` - If the type is not one of: TEXT, BOOLEAN, or NUMBER. + +#### to\_dict + +```python +def to_dict() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utility_model.py#L81) + +Convert the input parameter to a dictionary representation. + +**Returns**: + +- `dict` - A dictionary containing the input parameter's name, description, + and type (as a string value). + +#### utility\_tool + +```python +def utility_tool(name: Text, + description: Text, + inputs: List[UtilityModelInput] = None, + output_examples: Text = "", + status=AssetStatus.DRAFT) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utility_model.py#L92) + +Decorator for utility tool functions + +**Arguments**: + +- `name` - Name of the utility tool +- `description` - Description of what the utility tool does +- `inputs` - List of input parameters, must be UtilityModelInput objects +- `output_examples` - Examples of expected outputs +- `status` - Asset status + + +**Raises**: + +- `ValueError` - If name or description is empty +- `TypeError` - If inputs contains non-UtilityModelInput objects + +### UtilityModel Objects + +```python +class UtilityModel(Model, DeployableMixin) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utility_model.py#L134) + +Ready-to-use Utility Model. + +Note: Non-deployed utility models (status=DRAFT) will expire after 24 hours after creation. +Use the .deploy() method to make the model permanent. + +**Attributes**: + +- `id` _Text_ - ID of the Model +- `name` _Text_ - Name of the Model +- `code` _Union[Text, Callable]_ - code of the model. +- `description` _Text_ - description of the model. Defaults to "". +- `inputs` _List[UtilityModelInput]_ - inputs of the model. Defaults to []. +- `output_examples` _Text_ - output examples. Defaults to "". +- `api_key` _Text, optional_ - API key of the Model. Defaults to None. +- `supplier` _Union[Dict, Text, Supplier, int], optional_ - supplier of the asset. Defaults to "aiXplain". +- `version` _Text, optional_ - version of the model. Defaults to "1.0". +- `function` _Function, optional_ - model AI function. Defaults to None. +- `name`0 _bool, optional_ - Is the user subscribed. Defaults to False. +- `name`1 _Dict, optional_ - model price. Defaults to None. +- `name`2 _AssetStatus, optional_ - status of the model. Defaults to AssetStatus.DRAFT. +- `name`3 - Any additional Model info to be saved + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Optional[Text] = None, + code: Union[Text, Callable] = None, + description: Optional[Text] = None, + inputs: List[UtilityModelInput] = [], + output_examples: Text = "", + api_key: Optional[Text] = None, + supplier: Union[Dict, Text, Supplier, int] = "aiXplain", + version: Optional[Text] = None, + function: Optional[Function] = None, + is_subscribed: bool = False, + cost: Optional[Dict] = None, + status: AssetStatus = AssetStatus.DRAFT, + function_type: Optional[FunctionType] = FunctionType.UTILITY, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utility_model.py#L157) + +Initialize a new UtilityModel instance. + +**Arguments**: + +- `id` _Text_ - ID of the utility model. +- `name` _Optional[Text], optional_ - Name of the utility model. If not provided, + will be extracted from the code if decorated. Defaults to None. +- `code` _Union[Text, Callable], optional_ - Implementation code, either as a string + or a callable function. Defaults to None. +- `description` _Optional[Text], optional_ - Description of what the model does. + If not provided, will be extracted from the code if decorated. + Defaults to None. +- `inputs` _List[UtilityModelInput], optional_ - List of input parameters the + model accepts. If not provided, will be extracted from the code if + decorated. Defaults to []. +- `output_examples` _Text, optional_ - Examples of the model's expected outputs. + Defaults to "". +- `api_key` _Optional[Text], optional_ - API key for accessing the model. + Defaults to None. +- `supplier` _Union[Dict, Text, Supplier, int], optional_ - Supplier of the model. + Defaults to "aiXplain". +- `version` _Optional[Text], optional_ - Version of the model. Defaults to None. +- `function` _Optional[Function], optional_ - Function type. Must be + Function.UTILITIES. Defaults to None. +- `name`0 _bool, optional_ - Whether the user is subscribed. + Defaults to False. +- `name`1 _Optional[Dict], optional_ - Cost information for the model. + Defaults to None. +- `name`2 _AssetStatus, optional_ - Current status of the model. + Defaults to AssetStatus.DRAFT. +- `name`3 _Optional[FunctionType], optional_ - Type of the function. + Defaults to FunctionType.UTILITY. +- `name`4 - Any additional model info to be saved. + + +**Raises**: + +- `name`5 - If function is not Function.UTILITIES. + + +**Notes**: + + Non-deployed utility models (status=DRAFT) will expire after 24 hours. + Use the .deploy() method to make the model permanent. + +#### validate + +```python +def validate() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utility_model.py#L249) + +Validate the Utility Model. + +This method checks if the utility model exists in the backend and if the code is a string with s3://. +If not, it parses the code and updates the description and inputs and does the validation. +If yes, it just does the validation on the description and inputs. + +#### to\_dict + +```python +def to_dict() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utility_model.py#L300) + +Convert the utility model to a dictionary representation. + +This method creates a dictionary containing all the essential information +about the utility model, suitable for API requests or serialization. + +**Returns**: + +- `dict` - A dictionary containing: + - name (str): The model's name + - description (str): The model's description + - inputs (List[dict]): List of input parameters as dictionaries + - code (Union[str, Callable]): The model's implementation code + - function (str): The function type as a string value + - outputDescription (str): Examples of expected outputs + - status (str): Current status as a string value + +#### update + +```python +def update() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utility_model.py#L326) + +Update the Utility Model. + +This method validates the utility model and updates it in the backend. + +**Raises**: + +- `Exception` - If the update fails. + +#### save + +```python +def save() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utility_model.py#L363) + +Save the Utility Model. + +This method updates the utility model in the backend. + +#### delete + +```python +def delete() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utility_model.py#L370) + +Delete the Utility Model. + +This method deletes the utility model from the backend. + +#### \_\_repr\_\_ + +```python +def __repr__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utility_model.py#L391) + +Return a string representation of the UtilityModel instance. + +**Returns**: + +- `str` - A string in the format "UtilityModel: <name> by <supplier> (id=<id>)". + If supplier is a dictionary, uses supplier['name'], otherwise uses + supplier directly. + diff --git a/docs/api-reference/python/aixplain/modules/model/utils.md b/docs/api-reference/python/aixplain/modules/model/utils.md new file mode 100644 index 00000000..ff005850 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/model/utils.md @@ -0,0 +1,212 @@ +--- +sidebar_label: utils +title: aixplain.modules.model.utils +--- + +#### build\_payload + +```python +def build_payload(data: Union[Text, Dict], + parameters: Optional[Dict] = None, + stream: Optional[bool] = None) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utils.py#L10) + +Build a JSON payload for API requests. + +This function constructs a JSON payload by combining input data with optional +parameters and streaming configuration. It handles various input formats and +ensures proper JSON serialization. + +**Arguments**: + +- `data` _Union[Text, Dict]_ - The primary data to include in the payload. + Can be a string (which may be JSON) or a dictionary. +- `parameters` _Optional[Dict], optional_ - Additional parameters to include + in the payload. Defaults to None. +- `stream` _Optional[bool], optional_ - Whether to enable streaming for this + request. If provided, adds streaming configuration to parameters. + Defaults to None. + + +**Returns**: + +- `str` - A JSON string containing the complete payload with all parameters + and data properly formatted. + + +**Notes**: + + - If data is a string that can be parsed as JSON, it will be. + - If data is a number (after JSON parsing), it will be converted to string. + - The function ensures the result is a valid JSON string. + +#### call\_run\_endpoint + +```python +def call_run_endpoint(url: Text, api_key: Text, payload: Dict) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utils.py#L63) + +Call a model execution endpoint and handle the response. + +This function makes a POST request to a model execution endpoint, handles +various response scenarios, and provides appropriate error handling. + +**Arguments**: + +- `url` _Text_ - The endpoint URL to call. +- `api_key` _Text_ - API key for authentication. +- `payload` _Dict_ - The request payload to send. + + +**Returns**: + +- `Dict` - A response dictionary containing: + - status (str): "IN_PROGRESS", "SUCCESS", or "FAILED" + - completed (bool): Whether the request is complete + - url (str, optional): Polling URL for async requests + - data (Any, optional): Response data if available + - error_message (str, optional): Error message if failed + + +**Notes**: + + - For async operations, returns a polling URL in the 'url' field + - For failures, includes an error message and sets status to "FAILED" + - Handles both API errors and request exceptions + +#### parse\_code + +```python +def parse_code(code: Union[Text, Callable]) -> Tuple[Text, List, Text, Text] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utils.py#L127) + +Parse and process code for utility model creation. + +This function takes code input in various forms (callable, file path, URL, or +string) and processes it for use in a utility model. It extracts metadata, +validates the code structure, and prepares it for execution. + +**Arguments**: + +- `code` _Union[Text, Callable]_ - The code to parse. Can be: + - A callable function + - A file path (string) + - A URL (string) + - Raw code (string) + + +**Returns**: + + Tuple[Text, List, Text, Text]: A tuple containing: + - code (Text): The processed code, uploaded to storage + - inputs (List[UtilityModelInput]): List of extracted input parameters + - description (Text): Function description from docstring + - name (Text): Function name + + +**Raises**: + +- `Exception` - If the code doesn't have a main function +- `AssertionError` - If input types are not properly specified +- `Exception` - If an input type is not supported (must be int, float, bool, or str) + + +**Notes**: + + - The function requires a 'main' function in the code + - Input parameters must have type annotations + - Supported input types are: int, float, bool, str + - The code is uploaded to temporary storage for later use + +#### parse\_code\_decorated + +```python +def parse_code_decorated( + code: Union[Text, Callable]) -> Tuple[Text, List, Text, Text] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utils.py#L238) + +Parse and process code that may be decorated with @utility_tool. + +This function handles code that may be decorated with the @utility_tool +decorator, extracting metadata from either the decorator or the code itself. +It supports various input formats and provides robust parameter extraction. + +**Arguments**: + +- `code` _Union[Text, Callable]_ - The code to parse. Can be: + - A decorated callable function + - A non-decorated callable function + - A file path (string) + - A URL (string) + - Raw code (string) + + +**Returns**: + + Tuple[Text, List, Text, Text]: A tuple containing: + - code (Text): The processed code, uploaded to storage + - inputs (List[UtilityModelInput]): List of extracted input parameters + - description (Text): Function description from decorator or docstring + - name (Text): Function name from decorator or code + + +**Raises**: + +- `TypeError` - If code is a class or class instance +- `AssertionError` - If input types are not properly specified +- `Exception` - In various cases: + - If code doesn't have a function definition + - If code has invalid @utility_tool decorator + - If input type is not supported + - If code parsing fails + + +**Notes**: + + - Handles both decorated and non-decorated code + - For decorated code, extracts metadata from decorator + - For non-decorated code, falls back to code parsing + - Renames the function to 'main' for backend compatibility + - Supports TEXT, BOOLEAN, and NUMBER input types + - Uploads processed code to temporary storage + +#### is\_supported\_image\_type + +```python +def is_supported_image_type(value: str) -> bool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/model/utils.py#L462) + +Check if a file path or URL points to a supported image format. + +This function checks if the provided string ends with a supported image +file extension. The check is case-insensitive. + +**Arguments**: + +- `value` _str_ - The file path or URL to check. + + +**Returns**: + +- `bool` - True if the file has a supported image extension, False otherwise. + + +**Notes**: + + Supported image formats are: + - JPEG (.jpg, .jpeg) + - PNG (.png) + - GIF (.gif) + - BMP (.bmp) + - WebP (.webp) + diff --git a/docs/api-reference/python/aixplain/modules/pipeline/asset.md b/docs/api-reference/python/aixplain/modules/pipeline/asset.md new file mode 100644 index 00000000..5eb5c1d0 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/pipeline/asset.md @@ -0,0 +1,295 @@ +--- +sidebar_label: asset +title: aixplain.modules.pipeline.asset +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Thiago Castro Ferreira, Shreyas Sharma and Lucas Pavanelli +Date: November 25th 2024 +Description: + Pipeline Asset Class + +### Pipeline Objects + +```python +class Pipeline(Asset, DeployableMixin) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/asset.py#L39) + +Representing a custom pipeline that was created on the aiXplain Platform + +**Attributes**: + +- `id` _Text_ - ID of the Pipeline +- `name` _Text_ - Name of the Pipeline +- `api_key` _Text_ - Team API Key to run the Pipeline. +- `url` _Text, optional_ - running URL of platform. Defaults to config.BACKEND_URL. +- `supplier` _Text, optional_ - Pipeline supplier. Defaults to "aiXplain". +- `version` _Text, optional_ - version of the pipeline. Defaults to "1.0". +- `status` _AssetStatus, optional_ - Pipeline status. Defaults to AssetStatus.DRAFT. +- `**additional_info` - Any additional Pipeline info to be saved + +#### \_\_init\_\_ + +```python +def __init__(id: Text, + name: Text, + api_key: Text, + url: Text = config.BACKEND_URL, + supplier: Text = "aiXplain", + version: Text = "1.0", + status: AssetStatus = AssetStatus.DRAFT, + **additional_info) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/asset.py#L56) + +Create a Pipeline with the necessary information + +**Arguments**: + +- `id` _Text_ - ID of the Pipeline +- `name` _Text_ - Name of the Pipeline +- `api_key` _Text_ - Team API Key to run the Pipeline. +- `url` _Text, optional_ - running URL of platform. Defaults to config.BACKEND_URL. +- `supplier` _Text, optional_ - Pipeline supplier. Defaults to "aiXplain". +- `version` _Text, optional_ - version of the pipeline. Defaults to "1.0". +- `status` _AssetStatus, optional_ - Pipeline status. Defaults to AssetStatus.DRAFT. +- `**additional_info` - Any additional Pipeline info to be saved + +#### poll + +```python +def poll(poll_url: Text, + name: Text = "pipeline_process", + response_version: Text = "v2") -> Union[Dict, PipelineResponse] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/asset.py#L143) + +Poll the platform to check whether an asynchronous call is done. + +**Arguments**: + +- `poll_url` _Text_ - polling URL +- `name` _Text, optional_ - ID given to a call. Defaults to "pipeline_process". + + +**Returns**: + +- `Dict` - response obtained by polling call + +#### run + +```python +def run(data: Union[Text, Dict], + data_asset: Optional[Union[Text, Dict]] = None, + name: Text = "pipeline_process", + timeout: float = 20000.0, + wait_time: float = 1.0, + version: Optional[Text] = None, + response_version: Text = "v2", + **kwargs) -> Union[Dict, PipelineResponse] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/asset.py#L188) + +Run the pipeline synchronously and wait for results. + +This method executes the pipeline with the provided input data and waits +for completion. It handles both direct data input and data assets, with +support for polling and timeout. + +**Arguments**: + +- `data` _Union[Text, Dict]_ - The input data for the pipeline. Can be: + - A string (file path, URL, or raw data) + - A dictionary mapping node labels to input data +- `data_asset` _Optional[Union[Text, Dict]], optional_ - Data asset(s) to + process. Can be a single asset ID or a dict mapping node labels + to asset IDs. Defaults to None. +- `name` _Text, optional_ - Identifier for this pipeline run. Used for + logging. Defaults to "pipeline_process". +- `timeout` _float, optional_ - Maximum time in seconds to wait for + completion. Defaults to 20000.0. +- `wait_time` _float, optional_ - Initial time in seconds between polling + attempts. May increase over time. Defaults to 1.0. +- `version` _Optional[Text], optional_ - Specific pipeline version to run. + Defaults to None. +- `response_version` _Text, optional_ - Response format version ("v1" or + "v2"). Defaults to "v2". +- `**kwargs` - Additional keyword arguments passed to the pipeline. + + +**Returns**: + + Union[Dict, PipelineResponse]: If response_version is: + - "v1": Dictionary with status, error (if any), and elapsed time + - "v2": PipelineResponse object with structured response data + + +**Raises**: + +- `Exception` - If the pipeline execution fails, times out, or encounters + errors during polling. + + +**Notes**: + + - The method starts with run_async and then polls for completion + - wait_time may increase up to 60 seconds between polling attempts + - For v2 responses, use PipelineResponse methods to access results + +#### run\_async + +```python +def run_async(data: Union[Text, Dict], + data_asset: Optional[Union[Text, Dict]] = None, + name: Text = "pipeline_process", + batch_mode: bool = True, + version: Optional[Text] = None, + response_version: Text = "v2", + **kwargs) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/asset.py#L425) + +Runs asynchronously a pipeline call. + +**Arguments**: + +- `data` _Union[Text, Dict]_ - link to the input data +- `data_asset` _Optional[Union[Text, Dict]], optional_ - Data asset to be processed by the pipeline. Defaults to None. +- `name` _Text, optional_ - ID given to a call. Defaults to "pipeline_process". +- `batch_mode` _bool, optional_ - Whether to run the pipeline in batch mode or online. Defaults to True. +- `version` _Optional[Text], optional_ - Version of the pipeline. Defaults to None. +- `response_version` _Text, optional_ - Version of the response. Defaults to "v2". +- `kwargs` - A dictionary of keyword arguments. The keys are the argument names + + +**Returns**: + +- `Dict` - polling URL in response + +#### update + +```python +def update(pipeline: Union[Text, Dict], + save_as_asset: bool = False, + api_key: Optional[Text] = None, + name: Optional[Text] = None) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/asset.py#L510) + +Update Pipeline + +**Arguments**: + +- `pipeline` _Union[Text, Dict]_ - Pipeline as a Python dictionary or in a JSON file +- `save_as_asset` _bool, optional_ - Save as asset (True) or draft (False). Defaults to False. +- `api_key` _Optional[Text], optional_ - Team API Key to create the Pipeline. Defaults to None. + + +**Raises**: + +- `Exception` - Make sure the pipeline to be save is in a JSON file. + +#### delete + +```python +def delete() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/asset.py#L574) + +Delete this pipeline from the platform. + +This method permanently removes the pipeline from the aiXplain platform. +The operation cannot be undone. + +**Raises**: + +- `Exception` - If deletion fails, which can happen if: + - The pipeline doesn't exist + - The user doesn't have permission to delete it + - The API request fails + - The server returns a non-200 status code + + +**Notes**: + + - This operation is permanent and cannot be undone + - Only the pipeline owner can delete it + - Uses the team API key for authentication + +#### save + +```python +def save(pipeline: Optional[Union[Text, Dict]] = None, + save_as_asset: bool = False, + api_key: Optional[Text] = None) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/asset.py#L607) + +Update and Save Pipeline + +**Arguments**: + +- `pipeline` _Optional[Union[Text, Dict]]_ - Pipeline as a Python dictionary or in a JSON file +- `save_as_asset` _bool, optional_ - Save as asset (True) or draft (False). Defaults to False. +- `api_key` _Optional[Text], optional_ - Team API Key to create the Pipeline. Defaults to None. + + +**Raises**: + +- `Exception` - Make sure the pipeline to be save is in a JSON file. + +#### deploy + +```python +def deploy(api_key: Optional[Text] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/asset.py#L663) + +Deploy the Pipeline. + +This method overrides the deploy method in DeployableMixin to handle +Pipeline-specific deployment functionality. + +**Arguments**: + +- `api_key` _Optional[Text], optional_ - Team API Key to deploy the Pipeline. Defaults to None. + +#### \_\_repr\_\_ + +```python +def __repr__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/asset.py#L682) + +Return a string representation of the Pipeline instance. + +**Returns**: + +- `str` - A string in the format "Pipeline: <name> (id=<id>)". + diff --git a/docs/api-reference/python/aixplain/modules/pipeline/default.md b/docs/api-reference/python/aixplain/modules/pipeline/default.md new file mode 100644 index 00000000..e0a99c73 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/pipeline/default.md @@ -0,0 +1,45 @@ +--- +sidebar_label: default +title: aixplain.modules.pipeline.default +--- + +### DefaultPipeline Objects + +```python +class DefaultPipeline(PipelineAsset, DesignerPipeline) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/default.py#L6) + +DefaultPipeline is a subclass of PipelineAsset and DesignerPipeline. + +#### \_\_init\_\_ + +```python +def __init__(*args, **kwargs) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/default.py#L10) + +Initialize the DefaultPipeline. + +#### save + +```python +def save(*args, **kwargs) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/default.py#L17) + +Save the DefaultPipeline. + +#### to\_dict + +```python +def to_dict() -> dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/default.py#L25) + +Convert the DefaultPipeline to a dictionary. + diff --git a/docs/api-reference/python/aixplain/modules/pipeline/designer/base.md b/docs/api-reference/python/aixplain/modules/pipeline/designer/base.md new file mode 100644 index 00000000..9238e24b --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/pipeline/designer/base.md @@ -0,0 +1,138 @@ +--- +sidebar_label: base +title: aixplain.modules.pipeline.designer.base +--- + +### Param Objects + +```python +class Param(Serializable) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/base.py#L28) + +Param class, this class will be used to create the parameters of the node. + +#### attach\_to + +```python +def attach_to(node: "Node") -> "Param" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/base.py#L58) + +Attach the param to the node. + +**Arguments**: + +- `node`: the node + +**Returns**: + +the param + +#### link + +```python +def link(to_param: "Param") -> "Param" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/base.py#L75) + +Link the output of the param to the input of another param. + +**Arguments**: + +- `to_param`: the input param + +**Returns**: + +the param + +#### back\_link + +```python +def back_link(from_param: "Param") -> "Param" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/base.py#L86) + +Link the input of the param to the output of another param. + +**Arguments**: + +- `from_param`: the output param + +**Returns**: + +the param + +### Link Objects + +```python +class Link(Serializable) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/base.py#L121) + +Link class, this class will be used to link the output of the node to the +input of another node. + +#### attach\_to + +```python +def attach_to(pipeline: "DesignerPipeline") +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/base.py#L204) + +Attach the link to the pipeline. + +**Arguments**: + +- `pipeline`: the pipeline + +### ParamProxy Objects + +```python +class ParamProxy(Serializable) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/base.py#L236) + +#### special\_prompt\_handling + +```python +def special_prompt_handling(code: str, value: str) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/base.py#L285) + +This method will handle the special prompt handling for asset nodes +having `text-generation` function type. + +### Node Objects + +```python +class Node(Generic[TI, TO], Serializable) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/base.py#L357) + +Node class is the base class for all the nodes in the pipeline. This class +will be used to create the nodes and link them together. + +#### attach\_to + +```python +def attach_to(pipeline: "DesignerPipeline") +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/base.py#L390) + +Attach the node to the pipeline. + +**Arguments**: + +- `pipeline`: the pipeline + diff --git a/docs/api-reference/python/aixplain/modules/pipeline/designer/enums.md b/docs/api-reference/python/aixplain/modules/pipeline/designer/enums.md new file mode 100644 index 00000000..0b94d0da --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/pipeline/designer/enums.md @@ -0,0 +1,5 @@ +--- +sidebar_label: enums +title: aixplain.modules.pipeline.designer.enums +--- + diff --git a/docs/api-reference/python/aixplain/modules/pipeline/designer/init.md b/docs/api-reference/python/aixplain/modules/pipeline/designer/init.md new file mode 100644 index 00000000..10863e50 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/pipeline/designer/init.md @@ -0,0 +1,6 @@ +--- +draft: true +sidebar_label: designer +title: aixplain.modules.pipeline.designer +--- + diff --git a/docs/api-reference/python/aixplain/modules/pipeline/designer/mixins.md b/docs/api-reference/python/aixplain/modules/pipeline/designer/mixins.md new file mode 100644 index 00000000..2e49a16c --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/pipeline/designer/mixins.md @@ -0,0 +1,107 @@ +--- +sidebar_label: mixins +title: aixplain.modules.pipeline.designer.mixins +--- + +### LinkableMixin Objects + +```python +class LinkableMixin() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/mixins.py#L5) + +Linkable mixin class, this class will be used to link the output of the +node to the input of another node. + +This class will be used to link the output of the node to the input of +another node. + +#### link + +```python +def link(to_node: Node, from_param: Union[str, Param], + to_param: Union[str, Param]) -> Link +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/mixins.py#L14) + +Link the output of the node to the input of another node. This method + +will link the output of the node to the input of another node. + +**Arguments**: + +- `to_node`: the node to link to the output +- `from_param`: the output parameter or the code of the output +parameter +- `to_param`: the input parameter or the code of the input parameter + +**Returns**: + +the link + +### RoutableMixin Objects + +```python +class RoutableMixin() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/mixins.py#L39) + +Routable mixin class, this class will be used to route the input data to +different nodes based on the input data type. + +#### route + +```python +def route(*params: Param) -> Node +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/mixins.py#L45) + +Route the input data to different nodes based on the input data type. + +This method will automatically link the input data to the output data +of the node. + +**Arguments**: + +- `params`: the output parameters + +**Returns**: + +the router node + +### OutputableMixin Objects + +```python +class OutputableMixin() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/mixins.py#L63) + +Outputable mixin class, this class will be used to link the output of the +node to the output node of the pipeline. + +#### use\_output + +```python +def use_output(param: Union[str, Param]) -> Node +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/mixins.py#L69) + +Use the output of the node as the output of the pipeline. + +This method will automatically link the output of the node to the +output node of the pipeline. + +**Arguments**: + +- `param`: the output parameter or the code of the output parameter + +**Returns**: + +the output node + diff --git a/docs/api-reference/python/aixplain/modules/pipeline/designer/nodes.md b/docs/api-reference/python/aixplain/modules/pipeline/designer/nodes.md new file mode 100644 index 00000000..266b87c4 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/pipeline/designer/nodes.md @@ -0,0 +1,155 @@ +--- +sidebar_label: nodes +title: aixplain.modules.pipeline.designer.nodes +--- + +### AssetNode Objects + +```python +class AssetNode(Node[TI, TO], LinkableMixin, OutputableMixin) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/nodes.py#L26) + +Asset node class, this node will be used to fetch the asset from the +aixplain platform and use it in the pipeline. + +`assetId` is required and will be used to fetch the asset from the +aixplain platform. + +Input and output parameters will be automatically added based on the +asset function spec. + +### Input Objects + +```python +class Input(Node[InputInputs, InputOutputs], LinkableMixin, RoutableMixin) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/nodes.py#L184) + +Input node class, this node will be used to input the data to the +pipeline. + +Input nodes has only one output parameter called `input`. + +`data` is a special convenient parameter that will be uploaded to the +aixplain platform and the link will be passed as the input to the node. + +### Output Objects + +```python +class Output(Node[OutputInputs, OutputOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/nodes.py#L236) + +Output node class, this node will be used to output the result of the +pipeline. + +Output nodes has only one input parameter called `output`. + +### Script Objects + +```python +class Script(Node[TI, TO], LinkableMixin, OutputableMixin) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/nodes.py#L259) + +Script node class, this node will be used to run a script on the input +data. + +`script_path` is a special convenient parameter that will be uploaded to +the aixplain platform and the link will be passed as the input to the node. + +### Route Objects + +```python +class Route(Serializable) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/nodes.py#L299) + +Route class, this class will be used to route the input data to different +nodes based on the input data type. + +#### \_\_init\_\_ + +```python +def __init__(value: DataType, path: List[Union[Node, int]], + operation: Operation, type: RouteType, **kwargs) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/nodes.py#L310) + +Post init method to convert the nodes to node numbers if they are +nodes. + +### Router Objects + +```python +class Router(Node[RouterInputs, RouterOutputs], LinkableMixin) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/nodes.py#L352) + +Router node class, this node will be used to route the input data to +different nodes based on the input data type. + +### Decision Objects + +```python +class Decision(Node[DecisionInputs, DecisionOutputs], LinkableMixin) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/nodes.py#L391) + +Decision node class, this node will be used to make decisions based on +the input data. + +### BaseSegmentor Objects + +```python +class BaseSegmentor(AssetNode[TI, TO]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/nodes.py#L441) + +Segmentor node class, this node will be used to segment the input data +into smaller fragments for much easier and efficient processing. + +### BareSegmentor Objects + +```python +class BareSegmentor(BaseSegmentor[SegmentorInputs, SegmentorOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/nodes.py#L463) + +Segmentor node class, this node will be used to segment the input data +into smaller fragments for much easier and efficient processing. + +### BaseReconstructor Objects + +```python +class BaseReconstructor(AssetNode[TI, TO]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/nodes.py#L475) + +Reconstructor node class, this node will be used to reconstruct the +output of the segmented lines of execution. + +### BareReconstructor Objects + +```python +class BareReconstructor(BaseReconstructor[ReconstructorInputs, + ReconstructorOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/nodes.py#L493) + +Reconstructor node class, this node will be used to reconstruct the +output of the segmented lines of execution. + diff --git a/docs/api-reference/python/aixplain/modules/pipeline/designer/pipeline.md b/docs/api-reference/python/aixplain/modules/pipeline/designer/pipeline.md new file mode 100644 index 00000000..1e8edc01 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/pipeline/designer/pipeline.md @@ -0,0 +1,474 @@ +--- +sidebar_label: pipeline +title: aixplain.modules.pipeline.designer.pipeline +--- + +### DesignerPipeline Objects + +```python +class DesignerPipeline(Serializable) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L26) + +#### add\_node + +```python +def add_node(node: Node) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L35) + +Add a node to the current pipeline. + +This method will take care of setting the pipeline instance to the +node and setting the node number if it's not set. + +**Arguments**: + +- `node`: the node + +**Returns**: + +the node + +#### add\_nodes + +```python +def add_nodes(*nodes: Node) -> List[Node] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L47) + +Add multiple nodes to the current pipeline. + +**Arguments**: + +- `nodes`: the nodes + +**Returns**: + +the nodes + +#### add\_link + +```python +def add_link(link: Link) -> Link +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L56) + +Add a link to the current pipeline. + +**Arguments**: + +- `link`: the link + +**Returns**: + +the link + +#### serialize + +```python +def serialize() -> dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L64) + +Serialize the pipeline to a dictionary. This method will serialize the + +pipeline to a dictionary. + +**Returns**: + +the pipeline as a dictionary + +#### validate\_nodes + +```python +def validate_nodes() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L89) + +Validate the linkage of the pipeline. This method will validate the + +linkage of the pipeline by applying the following checks: +- All input nodes are linked out +- All output nodes are linked in +- All other nodes are linked in and out + +**Raises**: + +- `ValueError`: if the pipeline is not valid + +#### is\_param\_linked + +```python +def is_param_linked(node, param) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L129) + +Check if the param is linked to another node. This method will check + +if the param is linked to another node. + +**Arguments**: + +- `node`: the node +- `param`: the param + +**Returns**: + +True if the param is linked, False otherwise + +#### is\_param\_set + +```python +def is_param_set(node, param) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L143) + +Check if the param is set. This method will check if the param is set + +or linked to another node. + +**Arguments**: + +- `node`: the node +- `param`: the param + +**Returns**: + +True if the param is set, False otherwise + +#### special\_prompt\_validation + +```python +def special_prompt_validation(node: Node) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L153) + +This method will handle the special rule for asset nodes having + +`text-generation` function type where if any prompt variable exists +then the `text` param is not required but the prompt param are. + +**Arguments**: + +- `node`: the node + +**Raises**: + +- `ValueError`: if the pipeline is not valid + +#### validate\_params + +```python +def validate_params() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L171) + +This method will check if all required params are either set or linked + +**Raises**: + +- `ValueError`: if the pipeline is not valid + +#### validate + +```python +def validate() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L183) + +Validate the pipeline. This method will validate the pipeline by + +series of checks: +- Validate all nodes are linked correctly +- Validate all required params are set or linked + +Any other validation checks can be added here. + +**Raises**: + +- `ValueError`: if the pipeline is not valid + +#### get\_link + +```python +def get_link(from_node: int, to_node: int) -> Link +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L197) + +Get the link between two nodes. This method will return the link + +between two nodes. + +**Arguments**: + +- `from_node`: the from node number +- `to_node`: the to node number + +**Returns**: + +the link + +#### get\_node + +```python +def get_node(node_number: int) -> Node +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L211) + +Get the node by its number. This method will return the node with the + +given number. + +**Arguments**: + +- `node_number`: the node number + +**Returns**: + +the node + +#### auto\_infer + +```python +def auto_infer() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L221) + +Automatically infer the data types of the nodes in the pipeline. +This method will automatically infer the data types of the nodes in the +pipeline by traversing the pipeline and setting the data types of the +nodes based on the data types of the connected nodes. + +#### asset + +```python +def asset(asset_id: str, + *args, + asset_class: Type[T] = AssetNode, + **kwargs) -> T +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L231) + +Shortcut to create an asset node for the current pipeline. + +All params will be passed as keyword arguments to the node +constructor. + +**Arguments**: + +- `kwargs`: keyword arguments + +**Returns**: + +the node + +#### utility + +```python +def utility(asset_id: str, + *args, + asset_class: Type[T] = Utility, + **kwargs) -> T +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L242) + +Shortcut to create an utility nodes for the current pipeline. + +All params will be passed as keyword arguments to the node +constructor. + +**Arguments**: + +- `kwargs`: keyword arguments + +**Returns**: + +the node + +#### decision + +```python +def decision(*args, **kwargs) -> Decision +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L256) + +Shortcut to create an decision node for the current pipeline. + +All params will be passed as keyword arguments to the node +constructor. + +**Arguments**: + +- `kwargs`: keyword arguments + +**Returns**: + +the node + +#### script + +```python +def script(*args, **kwargs) -> Script +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L267) + +Shortcut to create an script node for the current pipeline. + +All params will be passed as keyword arguments to the node +constructor. + +**Arguments**: + +- `kwargs`: keyword arguments + +**Returns**: + +the node + +#### input + +```python +def input(*args, **kwargs) -> Input +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L278) + +Shortcut to create an input node for the current pipeline. + +All params will be passed as keyword arguments to the node +constructor. + +**Arguments**: + +- `kwargs`: keyword arguments + +**Returns**: + +the node + +#### output + +```python +def output(*args, **kwargs) -> Output +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L289) + +Shortcut to create an output node for the current pipeline. + +All params will be passed as keyword arguments to the node +constructor. + +**Arguments**: + +- `kwargs`: keyword arguments + +**Returns**: + +the node + +#### router + +```python +def router(routes: Tuple[DataType, Node], *args, **kwargs) -> Router +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L300) + +Shortcut to create an decision node for the current pipeline. + +All params will be passed as keyword arguments to the node +constructor. The routes will be handled specially and will be +converted to Route instances in a convenient way. + +**Arguments**: + +- `routes`: the routes +- `kwargs`: keyword arguments + +**Returns**: + +the node + +#### bare\_reconstructor + +```python +def bare_reconstructor(*args, **kwargs) -> BareReconstructor +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L322) + +Shortcut to create an reconstructor node for the current pipeline. + +All params will be passed as keyword arguments to the node +constructor. + +**Arguments**: + +- `kwargs`: keyword arguments + +**Returns**: + +the node + +#### bare\_segmentor + +```python +def bare_segmentor(*args, **kwargs) -> BareSegmentor +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L333) + +Shortcut to create an segmentor node for the current pipeline. + +All params will be passed as keyword arguments to the node +constructor. + +**Arguments**: + +- `kwargs`: keyword arguments + +**Returns**: + +the node + +#### metric + +```python +def metric(*args, **kwargs) -> BareMetric +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/pipeline.py#L344) + +Shortcut to create an metric node for the current pipeline. + +All params will be passed as keyword arguments to the node +constructor. + +**Arguments**: + +- `kwargs`: keyword arguments + +**Returns**: + +the node + diff --git a/docs/api-reference/python/aixplain/modules/pipeline/designer/utils.md b/docs/api-reference/python/aixplain/modules/pipeline/designer/utils.md new file mode 100644 index 00000000..2130e998 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/pipeline/designer/utils.md @@ -0,0 +1,23 @@ +--- +sidebar_label: utils +title: aixplain.modules.pipeline.designer.utils +--- + +#### find\_prompt\_params + +```python +def find_prompt_params(prompt: str) -> List[str] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/designer/utils.py#L5) + +This method will find the prompt parameters in the prompt string. + +**Arguments**: + +- `prompt`: the prompt string + +**Returns**: + +list of prompt parameters + diff --git a/docs/api-reference/python/aixplain/modules/pipeline/init.md b/docs/api-reference/python/aixplain/modules/pipeline/init.md new file mode 100644 index 00000000..e8484be3 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/pipeline/init.md @@ -0,0 +1,6 @@ +--- +draft: true +sidebar_label: pipeline +title: aixplain.modules.pipeline +--- + diff --git a/docs/api-reference/python/aixplain/modules/pipeline/pipeline.md b/docs/api-reference/python/aixplain/modules/pipeline/pipeline.md new file mode 100644 index 00000000..50798d85 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/pipeline/pipeline.md @@ -0,0 +1,2925 @@ +--- +sidebar_label: pipeline +title: aixplain.modules.pipeline.pipeline +--- + +### ObjectDetection Objects + +```python +class ObjectDetection(AssetNode[ObjectDetectionInputs, + ObjectDetectionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L28) + +Object Detection is a computer vision technology that identifies and locates +objects within an image, typically by drawing bounding boxes around the +detected objects and classifying them into predefined categories. + + InputType: video + OutputType: text + +### TextEmbedding Objects + +```python +class TextEmbedding(AssetNode[TextEmbeddingInputs, TextEmbeddingOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L68) + +Text embedding is a process that converts text into numerical vectors, +capturing the semantic meaning and contextual relationships of words or +phrases, enabling machines to understand and analyze natural language more +effectively. + + InputType: text + OutputType: text + +### SemanticSegmentation Objects + +```python +class SemanticSegmentation(AssetNode[SemanticSegmentationInputs, + SemanticSegmentationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L103) + +Semantic segmentation is a computer vision process that involves classifying +each pixel in an image into a predefined category, effectively partitioning the +image into meaningful segments based on the objects or regions they represent. + + InputType: image + OutputType: label + +### ReferencelessAudioGenerationMetric Objects + +```python +class ReferencelessAudioGenerationMetric( + BaseMetric[ReferencelessAudioGenerationMetricInputs, + ReferencelessAudioGenerationMetricOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L141) + +The Referenceless Audio Generation Metric is a tool designed to evaluate the +quality of generated audio content without the need for a reference or original +audio sample for comparison. + + InputType: text + OutputType: text + +### ScriptExecution Objects + +```python +class ScriptExecution(AssetNode[ScriptExecutionInputs, + ScriptExecutionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L177) + +Script Execution refers to the process of running a set of programmed +instructions or code within a computing environment, enabling the automated +performance of tasks, calculations, or operations as defined by the script. + + InputType: text + OutputType: text + +### ImageImpainting Objects + +```python +class ImageImpainting(AssetNode[ImageImpaintingInputs, + ImageImpaintingOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L211) + +Image inpainting is a process that involves filling in missing or damaged parts +of an image in a way that is visually coherent and seamlessly blends with the +surrounding areas, often using advanced algorithms and techniques to restore +the image to its original or intended appearance. + + InputType: image + OutputType: image + +### ImageEmbedding Objects + +```python +class ImageEmbedding(AssetNode[ImageEmbeddingInputs, ImageEmbeddingOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L248) + +Image Embedding is a process that transforms an image into a fixed-dimensional +vector representation, capturing its essential features and enabling efficient +comparison, retrieval, and analysis in various machine learning and computer +vision tasks. + + InputType: image + OutputType: text + +### MetricAggregation Objects + +```python +class MetricAggregation(BaseMetric[MetricAggregationInputs, + MetricAggregationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L283) + +Metric Aggregation is a function that computes and summarizes numerical data by +applying statistical operations, such as averaging, summing, or finding the +minimum and maximum values, to provide insights and facilitate analysis of +large datasets. + + InputType: text + OutputType: text + +### SpeechTranslation Objects + +```python +class SpeechTranslation(AssetNode[SpeechTranslationInputs, + SpeechTranslationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L328) + +Speech Translation is a technology that converts spoken language in real-time +from one language to another, enabling seamless communication between speakers +of different languages. + + InputType: audio + OutputType: text + +### DepthEstimation Objects + +```python +class DepthEstimation(AssetNode[DepthEstimationInputs, + DepthEstimationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L364) + +Depth estimation is a computational process that determines the distance of +objects from a viewpoint, typically using visual data from cameras or sensors +to create a three-dimensional understanding of a scene. + + InputType: image + OutputType: text + +### NoiseRemoval Objects + +```python +class NoiseRemoval(AssetNode[NoiseRemovalInputs, NoiseRemovalOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L398) + +Noise Removal is a process that involves identifying and eliminating unwanted +random variations or disturbances from an audio signal to enhance the clarity +and quality of the underlying information. + + InputType: audio + OutputType: audio + +### Diacritization Objects + +```python +class Diacritization(AssetNode[DiacritizationInputs, DiacritizationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L438) + +Adds diacritical marks to text, essential for languages where meaning can +change based on diacritics. + + InputType: text + OutputType: text + +### AudioTranscriptAnalysis Objects + +```python +class AudioTranscriptAnalysis(AssetNode[AudioTranscriptAnalysisInputs, + AudioTranscriptAnalysisOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L479) + +Analyzes transcribed audio data for insights, patterns, or specific information +extraction. + + InputType: audio + OutputType: text + +### ExtractAudioFromVideo Objects + +```python +class ExtractAudioFromVideo(AssetNode[ExtractAudioFromVideoInputs, + ExtractAudioFromVideoOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L512) + +Isolates and extracts audio tracks from video files, aiding in audio analysis +or transcription tasks. + + InputType: video + OutputType: audio + +### AudioReconstruction Objects + +```python +class AudioReconstruction(BaseReconstructor[AudioReconstructionInputs, + AudioReconstructionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L545) + +Audio Reconstruction is the process of restoring or recreating audio signals +from incomplete, damaged, or degraded recordings to achieve a high-quality, +accurate representation of the original sound. + + InputType: audio + OutputType: audio + +### ClassificationMetric Objects + +```python +class ClassificationMetric(BaseMetric[ClassificationMetricInputs, + ClassificationMetricOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L587) + +A Classification Metric is a quantitative measure used to evaluate the quality +and effectiveness of classification models. + + InputType: text + OutputType: text + +### TextGenerationMetric Objects + +```python +class TextGenerationMetric(BaseMetric[TextGenerationMetricInputs, + TextGenerationMetricOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L626) + +A Text Generation Metric is a quantitative measure used to evaluate the quality +and effectiveness of text produced by natural language processing models, often +assessing aspects such as coherence, relevance, fluency, and adherence to given +prompts or instructions. + + InputType: text + OutputType: text + +### TextSpamDetection Objects + +```python +class TextSpamDetection(AssetNode[TextSpamDetectionInputs, + TextSpamDetectionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L667) + +Identifies and filters out unwanted or irrelevant text content, ideal for +moderating user-generated content or ensuring quality in communication +platforms. + + InputType: text + OutputType: label + +### TextToImageGeneration Objects + +```python +class TextToImageGeneration(AssetNode[TextToImageGenerationInputs, + TextToImageGenerationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L701) + +Creates a visual representation based on textual input, turning descriptions +into pictorial forms. Used in creative processes and content generation. + + InputType: text + OutputType: image + +### VoiceCloning Objects + +```python +class VoiceCloning(AssetNode[VoiceCloningInputs, VoiceCloningOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L746) + +Replicates a person's voice based on a sample, allowing for the generation of +speech in that person's tone and style. Used cautiously due to ethical +considerations. + + InputType: text + OutputType: audio + +### TextSegmenation Objects + +```python +class TextSegmenation(AssetNode[TextSegmenationInputs, + TextSegmenationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L782) + +Text Segmentation is the process of dividing a continuous text into meaningful +units, such as words, sentences, or topics, to facilitate easier analysis and +understanding. + + InputType: text + OutputType: text + +### BenchmarkScoringMt Objects + +```python +class BenchmarkScoringMt(AssetNode[BenchmarkScoringMtInputs, + BenchmarkScoringMtOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L820) + +Benchmark Scoring MT is a function designed to evaluate and score machine +translation systems by comparing their output against a set of predefined +benchmarks, thereby assessing their accuracy and performance. + + InputType: text + OutputType: label + +### ImageManipulation Objects + +```python +class ImageManipulation(AssetNode[ImageManipulationInputs, + ImageManipulationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L856) + +Image Manipulation refers to the process of altering or enhancing digital +images using various techniques and tools to achieve desired visual effects, +correct imperfections, or transform the image's appearance. + + InputType: image + OutputType: image + +### NamedEntityRecognition Objects + +```python +class NamedEntityRecognition(AssetNode[NamedEntityRecognitionInputs, + NamedEntityRecognitionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L898) + +Identifies and classifies named entities (e.g., persons, organizations, +locations) within text. Useful for information extraction, content tagging, and +search enhancements. + + InputType: text + OutputType: label + +### OffensiveLanguageIdentification Objects + +```python +class OffensiveLanguageIdentification( + AssetNode[OffensiveLanguageIdentificationInputs, + OffensiveLanguageIdentificationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L938) + +Detects language or phrases that might be considered offensive, aiding in +content moderation and creating respectful user interactions. + + InputType: text + OutputType: label + +### Search Objects + +```python +class Search(AssetNode[SearchInputs, SearchOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L971) + +An algorithm that identifies and returns data or items that match particular +keywords or conditions from a dataset. A fundamental tool for databases and +websites. + + InputType: text + OutputType: text + +### SentimentAnalysis Objects + +```python +class SentimentAnalysis(AssetNode[SentimentAnalysisInputs, + SentimentAnalysisOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1011) + +Determines the sentiment or emotion (e.g., positive, negative, neutral) of a +piece of text, aiding in understanding user feedback or market sentiment. + + InputType: text + OutputType: label + +### ImageColorization Objects + +```python +class ImageColorization(AssetNode[ImageColorizationInputs, + ImageColorizationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1044) + +Image colorization is a process that involves adding color to grayscale images, +transforming them from black-and-white to full-color representations, often +using advanced algorithms and machine learning techniques to predict and apply +the appropriate hues and shades. + + InputType: image + OutputType: image + +### SpeechClassification Objects + +```python +class SpeechClassification(AssetNode[SpeechClassificationInputs, + SpeechClassificationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1085) + +Categorizes audio clips based on their content, aiding in content organization +and targeted actions. + + InputType: audio + OutputType: label + +### DialectDetection Objects + +```python +class DialectDetection(AssetNode[DialectDetectionInputs, + DialectDetectionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1120) + +Identifies specific dialects within a language, aiding in localized content +creation or user experience personalization. + + InputType: audio + OutputType: text + +### VideoLabelDetection Objects + +```python +class VideoLabelDetection(AssetNode[VideoLabelDetectionInputs, + VideoLabelDetectionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1155) + +Identifies and tags objects, scenes, or activities within a video. Useful for +content indexing and recommendation systems. + + InputType: video + OutputType: label + +### SpeechSynthesis Objects + +```python +class SpeechSynthesis(AssetNode[SpeechSynthesisInputs, + SpeechSynthesisOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1200) + +Generates human-like speech from written text. Ideal for text-to-speech +applications, audiobooks, and voice assistants. + + InputType: text + OutputType: audio + +### SplitOnSilence Objects + +```python +class SplitOnSilence(AssetNode[SplitOnSilenceInputs, SplitOnSilenceOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1233) + +The "Split On Silence" function divides an audio recording into separate +segments based on periods of silence, allowing for easier editing and analysis +of individual sections. + + InputType: audio + OutputType: audio + +### ExpressionDetection Objects + +```python +class ExpressionDetection(AssetNode[ExpressionDetectionInputs, + ExpressionDetectionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1267) + +Expression Detection is the process of identifying and analyzing facial +expressions to interpret emotions or intentions using AI and computer vision +techniques. + + InputType: text + OutputType: label + +### AutoMaskGeneration Objects + +```python +class AutoMaskGeneration(AssetNode[AutoMaskGenerationInputs, + AutoMaskGenerationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1301) + +Auto-mask generation refers to the automated process of creating masks in image +processing or computer vision, typically for segmentation tasks. A mask is a +binary or multi-class image that labels different parts of an image, usually +separating the foreground (objects of interest) from the background, or +identifying specific object classes in an image. + + InputType: image + OutputType: label + +### DocumentImageParsing Objects + +```python +class DocumentImageParsing(AssetNode[DocumentImageParsingInputs, + DocumentImageParsingOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1337) + +Document Image Parsing is the process of analyzing and converting scanned or +photographed images of documents into structured, machine-readable formats by +identifying and extracting text, layout, and other relevant information. + + InputType: image + OutputType: text + +### EntityLinking Objects + +```python +class EntityLinking(AssetNode[EntityLinkingInputs, EntityLinkingOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1375) + +Associates identified entities in the text with specific entries in a knowledge +base or database. + + InputType: text + OutputType: label + +### ReferencelessTextGenerationMetricDefault Objects + +```python +class ReferencelessTextGenerationMetricDefault( + BaseMetric[ReferencelessTextGenerationMetricDefaultInputs, + ReferencelessTextGenerationMetricDefaultOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1412) + +The Referenceless Text Generation Metric Default is a function designed to +evaluate the quality of generated text without relying on reference texts for +comparison. + + InputType: text + OutputType: text + +### FillTextMask Objects + +```python +class FillTextMask(AssetNode[FillTextMaskInputs, FillTextMaskOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1454) + +Completes missing parts of a text based on the context, ideal for content +generation or data augmentation tasks. + + InputType: text + OutputType: text + +### SubtitlingTranslation Objects + +```python +class SubtitlingTranslation(AssetNode[SubtitlingTranslationInputs, + SubtitlingTranslationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1495) + +Converts the text of subtitles from one language to another, ensuring context +and cultural nuances are maintained. Essential for global content distribution. + + InputType: text + OutputType: text + +### InstanceSegmentation Objects + +```python +class InstanceSegmentation(AssetNode[InstanceSegmentationInputs, + InstanceSegmentationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1528) + +Instance segmentation is a computer vision task that involves detecting and +delineating each distinct object within an image, assigning a unique label and +precise boundary to every individual instance of objects, even if they belong +to the same category. + + InputType: image + OutputType: label + +### VisemeGeneration Objects + +```python +class VisemeGeneration(AssetNode[VisemeGenerationInputs, + VisemeGenerationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1569) + +Viseme Generation is the process of creating visual representations of +phonemes, which are the distinct units of sound in speech, to synchronize lip +movements with spoken words in animations or virtual avatars. + + InputType: text + OutputType: label + +### AudioGenerationMetric Objects + +```python +class AudioGenerationMetric(BaseMetric[AudioGenerationMetricInputs, + AudioGenerationMetricOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1609) + +The Audio Generation Metric is a quantitative measure used to evaluate the +quality, accuracy, and overall performance of audio generated by artificial +intelligence systems, often considering factors such as fidelity, +intelligibility, and similarity to human-produced audio. + + InputType: text + OutputType: text + +### VideoUnderstanding Objects + +```python +class VideoUnderstanding(AssetNode[VideoUnderstandingInputs, + VideoUnderstandingOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1652) + +Video Understanding is the process of analyzing and interpreting video content +to extract meaningful information, such as identifying objects, actions, +events, and contextual relationships within the footage. + + InputType: video + OutputType: text + +### TextNormalization Objects + +```python +class TextNormalization(AssetNode[TextNormalizationInputs, + TextNormalizationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1690) + +Converts unstructured or non-standard textual data into a more readable and +uniform format, dealing with abbreviations, numerals, and other non-standard +words. + + InputType: text + OutputType: label + +### AsrQualityEstimation Objects + +```python +class AsrQualityEstimation(AssetNode[AsrQualityEstimationInputs, + AsrQualityEstimationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1726) + +ASR Quality Estimation is a process that evaluates the accuracy and reliability +of automatic speech recognition systems by analyzing their performance in +transcribing spoken language into text. + + InputType: text + OutputType: label + +### VoiceActivityDetection Objects + +```python +class VoiceActivityDetection(BaseSegmentor[VoiceActivityDetectionInputs, + VoiceActivityDetectionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1770) + +Determines when a person is speaking in an audio clip. It's an essential +preprocessing step for other audio-related tasks. + + InputType: audio + OutputType: audio + +### SpeechNonSpeechClassification Objects + +```python +class SpeechNonSpeechClassification( + AssetNode[SpeechNonSpeechClassificationInputs, + SpeechNonSpeechClassificationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1809) + +Differentiates between speech and non-speech audio segments. Great for editing +software and transcription services to exclude irrelevant audio. + + InputType: audio + OutputType: label + +### AudioTranscriptImprovement Objects + +```python +class AudioTranscriptImprovement(AssetNode[AudioTranscriptImprovementInputs, + AudioTranscriptImprovementOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1852) + +Refines and corrects transcriptions generated from audio data, improving +readability and accuracy. + + InputType: audio + OutputType: text + +### TextContentModeration Objects + +```python +class TextContentModeration(AssetNode[TextContentModerationInputs, + TextContentModerationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1891) + +Scans and identifies potentially harmful, offensive, or inappropriate textual +content, ensuring safer user environments. + + InputType: text + OutputType: label + +### EmotionDetection Objects + +```python +class EmotionDetection(AssetNode[EmotionDetectionInputs, + EmotionDetectionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1930) + +Identifies human emotions from text or audio, enhancing user experience in +chatbots or customer feedback analysis. + + InputType: text + OutputType: label + +### AudioForcedAlignment Objects + +```python +class AudioForcedAlignment(AssetNode[AudioForcedAlignmentInputs, + AudioForcedAlignmentOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L1973) + +Synchronizes phonetic and phonological text with the corresponding segments in +an audio file. Useful in linguistic research and detailed transcription tasks. + + InputType: audio + OutputType: audio + +### VideoContentModeration Objects + +```python +class VideoContentModeration(AssetNode[VideoContentModerationInputs, + VideoContentModerationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2008) + +Automatically reviews video content to detect and possibly remove inappropriate +or harmful material. Essential for user-generated content platforms. + + InputType: video + OutputType: label + +### ImageLabelDetection Objects + +```python +class ImageLabelDetection(AssetNode[ImageLabelDetectionInputs, + ImageLabelDetectionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2043) + +Identifies objects, themes, or topics within images, useful for image +categorization, search, and recommendation systems. + + InputType: image + OutputType: label + +### VideoForcedAlignment Objects + +```python +class VideoForcedAlignment(AssetNode[VideoForcedAlignmentInputs, + VideoForcedAlignmentOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2086) + +Aligns the transcription of spoken content in a video with its corresponding +timecodes, facilitating subtitle creation. + + InputType: video + OutputType: video + +### TextGeneration Objects + +```python +class TextGeneration(AssetNode[TextGenerationInputs, TextGenerationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2127) + +Creates coherent and contextually relevant textual content based on prompts or +certain parameters. Useful for chatbots, content creation, and data +augmentation. + + InputType: text + OutputType: text + +### TextClassification Objects + +```python +class TextClassification(AssetNode[TextClassificationInputs, + TextClassificationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2167) + +Categorizes text into predefined groups or topics, facilitating content +organization and targeted actions. + + InputType: text + OutputType: label + +### SpeechEmbedding Objects + +```python +class SpeechEmbedding(AssetNode[SpeechEmbeddingInputs, + SpeechEmbeddingOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2206) + +Transforms spoken content into a fixed-size vector in a high-dimensional space +that captures the content's essence. Facilitates tasks like speech recognition +and speaker verification. + + InputType: audio + OutputType: text + +### TopicClassification Objects + +```python +class TopicClassification(AssetNode[TopicClassificationInputs, + TopicClassificationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2246) + +Assigns categories or topics to a piece of text based on its content, +facilitating content organization and retrieval. + + InputType: text + OutputType: label + +### Translation Objects + +```python +class Translation(AssetNode[TranslationInputs, TranslationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2293) + +Converts text from one language to another while maintaining the original +message's essence and context. Crucial for global communication. + + InputType: text + OutputType: text + +### SpeechRecognition Objects + +```python +class SpeechRecognition(AssetNode[SpeechRecognitionInputs, + SpeechRecognitionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2334) + +Converts spoken language into written text. Useful for transcription services, +voice assistants, and applications requiring voice-to-text capabilities. + + InputType: audio + OutputType: text + +### Subtitling Objects + +```python +class Subtitling(AssetNode[SubtitlingInputs, SubtitlingOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2377) + +Generates accurate subtitles for videos, enhancing accessibility for diverse +audiences. + + InputType: audio + OutputType: text + +### ImageCaptioning Objects + +```python +class ImageCaptioning(AssetNode[ImageCaptioningInputs, + ImageCaptioningOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2410) + +Image Captioning is a process that involves generating a textual description of +an image, typically using machine learning models to analyze the visual content +and produce coherent and contextually relevant sentences that describe the +objects, actions, and scenes depicted in the image. + + InputType: image + OutputType: text + +### AudioLanguageIdentification Objects + +```python +class AudioLanguageIdentification(AssetNode[AudioLanguageIdentificationInputs, + AudioLanguageIdentificationOutputs] + ) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2445) + +Audio Language Identification is a process that involves analyzing an audio +recording to determine the language being spoken. + + InputType: audio + OutputType: label + +### VideoEmbedding Objects + +```python +class VideoEmbedding(AssetNode[VideoEmbeddingInputs, VideoEmbeddingOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2480) + +Video Embedding is a process that transforms video content into a fixed- +dimensional vector representation, capturing essential features and patterns to +facilitate tasks such as retrieval, classification, and recommendation. + + InputType: video + OutputType: embedding + +### AsrAgeClassification Objects + +```python +class AsrAgeClassification(AssetNode[AsrAgeClassificationInputs, + AsrAgeClassificationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2514) + +The ASR Age Classification function is designed to analyze audio recordings of +speech to determine the speaker's age group by leveraging automatic speech +recognition (ASR) technology and machine learning algorithms. + + InputType: audio + OutputType: label + +### AudioIntentDetection Objects + +```python +class AudioIntentDetection(AssetNode[AudioIntentDetectionInputs, + AudioIntentDetectionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2548) + +Audio Intent Detection is a process that involves analyzing audio signals to +identify and interpret the underlying intentions or purposes behind spoken +words, enabling systems to understand and respond appropriately to human +speech. + + InputType: audio + OutputType: label + +### LanguageIdentification Objects + +```python +class LanguageIdentification(AssetNode[LanguageIdentificationInputs, + LanguageIdentificationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2583) + +Detects the language in which a given text is written, aiding in multilingual +platforms or content localization. + + InputType: text + OutputType: text + +### Ocr Objects + +```python +class Ocr(AssetNode[OcrInputs, OcrOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2618) + +Converts images of typed, handwritten, or printed text into machine-encoded +text. Used in digitizing printed texts for data retrieval. + + InputType: image + OutputType: text + +### AsrGenderClassification Objects + +```python +class AsrGenderClassification(AssetNode[AsrGenderClassificationInputs, + AsrGenderClassificationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2651) + +The ASR Gender Classification function analyzes audio recordings to determine +and classify the speaker's gender based on their voice characteristics. + + InputType: audio + OutputType: label + +### LanguageIdentificationAudio Objects + +```python +class LanguageIdentificationAudio(AssetNode[LanguageIdentificationAudioInputs, + LanguageIdentificationAudioOutputs] + ) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2684) + +The Language Identification Audio function analyzes audio input to determine +and identify the language being spoken. + + InputType: audio + OutputType: label + +### BaseModel Objects + +```python +class BaseModel(AssetNode[BaseModelInputs, BaseModelOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2719) + +The Base-Model function serves as a foundational framework designed to provide +essential features and capabilities upon which more specialized or advanced +models can be built and customized. + + InputType: text + OutputType: text + +### Loglikelihood Objects + +```python +class Loglikelihood(AssetNode[LoglikelihoodInputs, LoglikelihoodOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2753) + +The Log Likelihood function measures the probability of observing the given +data under a specific statistical model by taking the natural logarithm of the +likelihood function, thereby transforming the product of probabilities into a +sum, which simplifies the process of optimization and parameter estimation. + + InputType: text + OutputType: number + +### ImageToVideoGeneration Objects + +```python +class ImageToVideoGeneration(AssetNode[ImageToVideoGenerationInputs, + ImageToVideoGenerationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2790) + +The Image To Video Generation function transforms a series of static images +into a cohesive, dynamic video sequence, often incorporating transitions, +effects, and synchronization with audio to create a visually engaging +narrative. + + InputType: image + OutputType: video + +### PartOfSpeechTagging Objects + +```python +class PartOfSpeechTagging(AssetNode[PartOfSpeechTaggingInputs, + PartOfSpeechTaggingOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2827) + +Part of Speech Tagging is a natural language processing task that involves +assigning each word in a sentence its corresponding part of speech, such as +noun, verb, adjective, or adverb, based on its role and context within the +sentence. + + InputType: text + OutputType: label + +### BenchmarkScoringAsr Objects + +```python +class BenchmarkScoringAsr(AssetNode[BenchmarkScoringAsrInputs, + BenchmarkScoringAsrOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2866) + +Benchmark Scoring ASR is a function that evaluates and compares the performance +of automatic speech recognition systems by analyzing their accuracy, speed, and +other relevant metrics against a standardized set of benchmarks. + + InputType: audio + OutputType: label + +### VisualQuestionAnswering Objects + +```python +class VisualQuestionAnswering(AssetNode[VisualQuestionAnsweringInputs, + VisualQuestionAnsweringOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2904) + +Visual Question Answering (VQA) is a task in artificial intelligence that +involves analyzing an image and providing accurate, contextually relevant +answers to questions posed about the visual content of that image. + + InputType: image + OutputType: video + +### DocumentInformationExtraction Objects + +```python +class DocumentInformationExtraction( + AssetNode[DocumentInformationExtractionInputs, + DocumentInformationExtractionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2938) + +Document Information Extraction is the process of automatically identifying, +extracting, and structuring relevant data from unstructured or semi-structured +documents, such as invoices, receipts, contracts, and forms, to facilitate +easier data management and analysis. + + InputType: image + OutputType: text + +### VideoGeneration Objects + +```python +class VideoGeneration(AssetNode[VideoGenerationInputs, + VideoGenerationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L2973) + +Produces video content based on specific inputs or datasets. Can be used for +simulations, animations, or even deepfake detection. + + InputType: text + OutputType: video + +### MultiClassImageClassification Objects + +```python +class MultiClassImageClassification( + AssetNode[MultiClassImageClassificationInputs, + MultiClassImageClassificationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3006) + +Multi Class Image Classification is a machine learning task where an algorithm +is trained to categorize images into one of several predefined classes or +categories based on their visual content. + + InputType: image + OutputType: label + +### StyleTransfer Objects + +```python +class StyleTransfer(AssetNode[StyleTransferInputs, StyleTransferOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3040) + +Style Transfer is a technique in artificial intelligence that applies the +visual style of one image (such as the brushstrokes of a famous painting) to +the content of another image, effectively blending the artistic elements of the +first image with the subject matter of the second. + + InputType: image + OutputType: image + +### MultiClassTextClassification Objects + +```python +class MultiClassTextClassification( + AssetNode[MultiClassTextClassificationInputs, + MultiClassTextClassificationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3077) + +Multi Class Text Classification is a natural language processing task that +involves categorizing a given text into one of several predefined classes or +categories based on its content. + + InputType: text + OutputType: label + +### IntentClassification Objects + +```python +class IntentClassification(AssetNode[IntentClassificationInputs, + IntentClassificationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3113) + +Intent Classification is a natural language processing task that involves +analyzing and categorizing user text input to determine the underlying purpose +or goal behind the communication, such as booking a flight, asking for weather +information, or setting a reminder. + + InputType: text + OutputType: label + +### MultiLabelTextClassification Objects + +```python +class MultiLabelTextClassification( + AssetNode[MultiLabelTextClassificationInputs, + MultiLabelTextClassificationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3150) + +Multi Label Text Classification is a natural language processing task where a +given text is analyzed and assigned multiple relevant labels or categories from +a predefined set, allowing for the text to belong to more than one category +simultaneously. + + InputType: text + OutputType: label + +### TextReconstruction Objects + +```python +class TextReconstruction(BaseReconstructor[TextReconstructionInputs, + TextReconstructionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3185) + +Text Reconstruction is a process that involves piecing together fragmented or +incomplete text data to restore it to its original, coherent form. + + InputType: text + OutputType: text + +### FactChecking Objects + +```python +class FactChecking(AssetNode[FactCheckingInputs, FactCheckingOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3220) + +Fact Checking is the process of verifying the accuracy and truthfulness of +information, statements, or claims by cross-referencing with reliable sources +and evidence. + + InputType: text + OutputType: label + +### InverseTextNormalization Objects + +```python +class InverseTextNormalization(AssetNode[InverseTextNormalizationInputs, + InverseTextNormalizationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3254) + +Inverse Text Normalization is the process of converting spoken or written +language in its normalized form, such as numbers, dates, and abbreviations, +back into their original, more complex or detailed textual representations. + + InputType: text + OutputType: label + +### TextToAudio Objects + +```python +class TextToAudio(AssetNode[TextToAudioInputs, TextToAudioOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3290) + +The Text to Audio function converts written text into spoken words, allowing +users to listen to the content instead of reading it. + + InputType: text + OutputType: audio + +### ImageCompression Objects + +```python +class ImageCompression(AssetNode[ImageCompressionInputs, + ImageCompressionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3325) + +Reduces the size of image files without significantly compromising their visual +quality. Useful for optimizing storage and improving webpage load times. + + InputType: image + OutputType: image + +### MultilingualSpeechRecognition Objects + +```python +class MultilingualSpeechRecognition( + AssetNode[MultilingualSpeechRecognitionInputs, + MultilingualSpeechRecognitionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3360) + +Multilingual Speech Recognition is a technology that enables the automatic +transcription of spoken language into text across multiple languages, allowing +for seamless communication and understanding in diverse linguistic contexts. + + InputType: audio + OutputType: text + +### TextGenerationMetricDefault Objects + +```python +class TextGenerationMetricDefault( + BaseMetric[TextGenerationMetricDefaultInputs, + TextGenerationMetricDefaultOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3400) + +The "Text Generation Metric Default" function provides a standard set of +evaluation metrics for assessing the quality and performance of text generation +models. + + InputType: text + OutputType: text + +### ReferencelessTextGenerationMetric Objects + +```python +class ReferencelessTextGenerationMetric( + BaseMetric[ReferencelessTextGenerationMetricInputs, + ReferencelessTextGenerationMetricOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3438) + +The Referenceless Text Generation Metric is a method for evaluating the quality +of generated text without requiring a reference text for comparison, often +leveraging models or algorithms to assess coherence, relevance, and fluency +based on intrinsic properties of the text itself. + + InputType: text + OutputType: text + +### AudioEmotionDetection Objects + +```python +class AudioEmotionDetection(AssetNode[AudioEmotionDetectionInputs, + AudioEmotionDetectionOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3475) + +Audio Emotion Detection is a technology that analyzes vocal characteristics and +patterns in audio recordings to identify and classify the emotional state of +the speaker. + + InputType: audio + OutputType: label + +### KeywordSpotting Objects + +```python +class KeywordSpotting(AssetNode[KeywordSpottingInputs, + KeywordSpottingOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3509) + +Keyword Spotting is a function that enables the detection and identification of +specific words or phrases within a stream of audio, often used in voice- +activated systems to trigger actions or commands based on recognized keywords. + + InputType: audio + OutputType: label + +### TextSummarization Objects + +```python +class TextSummarization(AssetNode[TextSummarizationInputs, + TextSummarizationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3549) + +Extracts the main points from a larger body of text, producing a concise +summary without losing the primary message. + + InputType: text + OutputType: text + +### SplitOnLinebreak Objects + +```python +class SplitOnLinebreak(BaseSegmentor[SplitOnLinebreakInputs, + SplitOnLinebreakOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3584) + +The "Split On Linebreak" function divides a given string into a list of +substrings, using linebreaks (newline characters) as the points of separation. + + InputType: text + OutputType: text + +### OtherMultipurpose Objects + +```python +class OtherMultipurpose(AssetNode[OtherMultipurposeInputs, + OtherMultipurposeOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3619) + +The "Other (Multipurpose)" function serves as a versatile category designed to +accommodate a wide range of tasks and activities that do not fit neatly into +predefined classifications, offering flexibility and adaptability for various +needs. + + InputType: text + OutputType: text + +### SpeakerDiarizationAudio Objects + +```python +class SpeakerDiarizationAudio(BaseSegmentor[SpeakerDiarizationAudioInputs, + SpeakerDiarizationAudioOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3662) + +Identifies individual speakers and their respective speech segments within an +audio clip. Ideal for multi-speaker recordings or conference calls. + + InputType: audio + OutputType: label + +### ImageContentModeration Objects + +```python +class ImageContentModeration(AssetNode[ImageContentModerationInputs, + ImageContentModerationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3697) + +Detects and filters out inappropriate or harmful images, essential for +platforms with user-generated visual content. + + InputType: image + OutputType: label + +### TextDenormalization Objects + +```python +class TextDenormalization(AssetNode[TextDenormalizationInputs, + TextDenormalizationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3738) + +Converts standardized or normalized text into its original, often more +readable, form. Useful in natural language generation tasks. + + InputType: text + OutputType: label + +### SpeakerDiarizationVideo Objects + +```python +class SpeakerDiarizationVideo(AssetNode[SpeakerDiarizationVideoInputs, + SpeakerDiarizationVideoOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3777) + +Segments a video based on different speakers, identifying when each individual +speaks. Useful for transcriptions and understanding multi-person conversations. + + InputType: video + OutputType: label + +### TextToVideoGeneration Objects + +```python +class TextToVideoGeneration(AssetNode[TextToVideoGenerationInputs, + TextToVideoGenerationOutputs]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3812) + +Text To Video Generation is a process that converts written descriptions or +scripts into dynamic, visual video content using advanced algorithms and +artificial intelligence. + + InputType: text + OutputType: video + +### Pipeline Objects + +```python +class Pipeline(DefaultPipeline) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3830) + +#### object\_detection + +```python +def object_detection(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ObjectDetection +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3831) + +Object Detection is a computer vision technology that identifies and locates +objects within an image, typically by drawing bounding boxes around the +detected objects and classifying them into predefined categories. + +#### text\_embedding + +```python +def text_embedding(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextEmbedding +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3839) + +Text embedding is a process that converts text into numerical vectors, +capturing the semantic meaning and contextual relationships of words or +phrases, enabling machines to understand and analyze natural language more +effectively. + +#### semantic\_segmentation + +```python +def semantic_segmentation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> SemanticSegmentation +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3848) + +Semantic segmentation is a computer vision process that involves classifying +each pixel in an image into a predefined category, effectively partitioning the +image into meaningful segments based on the objects or regions they represent. + +#### referenceless\_audio\_generation\_metric + +```python +def referenceless_audio_generation_metric( + asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ReferencelessAudioGenerationMetric +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3856) + +The Referenceless Audio Generation Metric is a tool designed to evaluate the +quality of generated audio content without the need for a reference or original +audio sample for comparison. + +#### script\_execution + +```python +def script_execution(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ScriptExecution +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3866) + +Script Execution refers to the process of running a set of programmed +instructions or code within a computing environment, enabling the automated +performance of tasks, calculations, or operations as defined by the script. + +#### image\_impainting + +```python +def image_impainting(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ImageImpainting +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3874) + +Image inpainting is a process that involves filling in missing or damaged parts +of an image in a way that is visually coherent and seamlessly blends with the +surrounding areas, often using advanced algorithms and techniques to restore +the image to its original or intended appearance. + +#### image\_embedding + +```python +def image_embedding(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ImageEmbedding +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3883) + +Image Embedding is a process that transforms an image into a fixed-dimensional +vector representation, capturing its essential features and enabling efficient +comparison, retrieval, and analysis in various machine learning and computer +vision tasks. + +#### metric\_aggregation + +```python +def metric_aggregation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> MetricAggregation +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3892) + +Metric Aggregation is a function that computes and summarizes numerical data by +applying statistical operations, such as averaging, summing, or finding the +minimum and maximum values, to provide insights and facilitate analysis of +large datasets. + +#### speech\_translation + +```python +def speech_translation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> SpeechTranslation +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3901) + +Speech Translation is a technology that converts spoken language in real-time +from one language to another, enabling seamless communication between speakers +of different languages. + +#### depth\_estimation + +```python +def depth_estimation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> DepthEstimation +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3909) + +Depth estimation is a computational process that determines the distance of +objects from a viewpoint, typically using visual data from cameras or sensors +to create a three-dimensional understanding of a scene. + +#### noise\_removal + +```python +def noise_removal(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> NoiseRemoval +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3917) + +Noise Removal is a process that involves identifying and eliminating unwanted +random variations or disturbances from an audio signal to enhance the clarity +and quality of the underlying information. + +#### diacritization + +```python +def diacritization(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> Diacritization +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3925) + +Adds diacritical marks to text, essential for languages where meaning can +change based on diacritics. + +#### audio\_transcript\_analysis + +```python +def audio_transcript_analysis(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> AudioTranscriptAnalysis +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3932) + +Analyzes transcribed audio data for insights, patterns, or specific information +extraction. + +#### extract\_audio\_from\_video + +```python +def extract_audio_from_video(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ExtractAudioFromVideo +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3939) + +Isolates and extracts audio tracks from video files, aiding in audio analysis +or transcription tasks. + +#### audio\_reconstruction + +```python +def audio_reconstruction(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> AudioReconstruction +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3946) + +Audio Reconstruction is the process of restoring or recreating audio signals +from incomplete, damaged, or degraded recordings to achieve a high-quality, +accurate representation of the original sound. + +#### classification\_metric + +```python +def classification_metric(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ClassificationMetric +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3954) + +A Classification Metric is a quantitative measure used to evaluate the quality +and effectiveness of classification models. + +#### text\_generation\_metric + +```python +def text_generation_metric(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextGenerationMetric +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3961) + +A Text Generation Metric is a quantitative measure used to evaluate the quality +and effectiveness of text produced by natural language processing models, often +assessing aspects such as coherence, relevance, fluency, and adherence to given +prompts or instructions. + +#### text\_spam\_detection + +```python +def text_spam_detection(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextSpamDetection +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3970) + +Identifies and filters out unwanted or irrelevant text content, ideal for +moderating user-generated content or ensuring quality in communication +platforms. + +#### text\_to\_image\_generation + +```python +def text_to_image_generation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextToImageGeneration +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3978) + +Creates a visual representation based on textual input, turning descriptions +into pictorial forms. Used in creative processes and content generation. + +#### voice\_cloning + +```python +def voice_cloning(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> VoiceCloning +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3985) + +Replicates a person's voice based on a sample, allowing for the generation of +speech in that person's tone and style. Used cautiously due to ethical +considerations. + +#### text\_segmenation + +```python +def text_segmenation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextSegmenation +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L3993) + +Text Segmentation is the process of dividing a continuous text into meaningful +units, such as words, sentences, or topics, to facilitate easier analysis and +understanding. + +#### benchmark\_scoring\_mt + +```python +def benchmark_scoring_mt(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> BenchmarkScoringMt +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4001) + +Benchmark Scoring MT is a function designed to evaluate and score machine +translation systems by comparing their output against a set of predefined +benchmarks, thereby assessing their accuracy and performance. + +#### image\_manipulation + +```python +def image_manipulation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ImageManipulation +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4009) + +Image Manipulation refers to the process of altering or enhancing digital +images using various techniques and tools to achieve desired visual effects, +correct imperfections, or transform the image's appearance. + +#### named\_entity\_recognition + +```python +def named_entity_recognition(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> NamedEntityRecognition +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4017) + +Identifies and classifies named entities (e.g., persons, organizations, +locations) within text. Useful for information extraction, content tagging, and +search enhancements. + +#### offensive\_language\_identification + +```python +def offensive_language_identification( + asset_id: Union[str, asset.Asset], *args, + **kwargs) -> OffensiveLanguageIdentification +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4025) + +Detects language or phrases that might be considered offensive, aiding in +content moderation and creating respectful user interactions. + +#### search + +```python +def search(asset_id: Union[str, asset.Asset], *args, **kwargs) -> Search +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4034) + +An algorithm that identifies and returns data or items that match particular +keywords or conditions from a dataset. A fundamental tool for databases and +websites. + +#### sentiment\_analysis + +```python +def sentiment_analysis(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> SentimentAnalysis +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4042) + +Determines the sentiment or emotion (e.g., positive, negative, neutral) of a +piece of text, aiding in understanding user feedback or market sentiment. + +#### image\_colorization + +```python +def image_colorization(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ImageColorization +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4049) + +Image colorization is a process that involves adding color to grayscale images, +transforming them from black-and-white to full-color representations, often +using advanced algorithms and machine learning techniques to predict and apply +the appropriate hues and shades. + +#### speech\_classification + +```python +def speech_classification(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> SpeechClassification +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4058) + +Categorizes audio clips based on their content, aiding in content organization +and targeted actions. + +#### dialect\_detection + +```python +def dialect_detection(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> DialectDetection +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4065) + +Identifies specific dialects within a language, aiding in localized content +creation or user experience personalization. + +#### video\_label\_detection + +```python +def video_label_detection(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> VideoLabelDetection +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4072) + +Identifies and tags objects, scenes, or activities within a video. Useful for +content indexing and recommendation systems. + +#### speech\_synthesis + +```python +def speech_synthesis(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> SpeechSynthesis +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4079) + +Generates human-like speech from written text. Ideal for text-to-speech +applications, audiobooks, and voice assistants. + +#### split\_on\_silence + +```python +def split_on_silence(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> SplitOnSilence +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4086) + +The "Split On Silence" function divides an audio recording into separate +segments based on periods of silence, allowing for easier editing and analysis +of individual sections. + +#### expression\_detection + +```python +def expression_detection(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ExpressionDetection +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4094) + +Expression Detection is the process of identifying and analyzing facial +expressions to interpret emotions or intentions using AI and computer vision +techniques. + +#### auto\_mask\_generation + +```python +def auto_mask_generation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> AutoMaskGeneration +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4102) + +Auto-mask generation refers to the automated process of creating masks in image +processing or computer vision, typically for segmentation tasks. A mask is a +binary or multi-class image that labels different parts of an image, usually +separating the foreground (objects of interest) from the background, or +identifying specific object classes in an image. + +#### document\_image\_parsing + +```python +def document_image_parsing(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> DocumentImageParsing +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4112) + +Document Image Parsing is the process of analyzing and converting scanned or +photographed images of documents into structured, machine-readable formats by +identifying and extracting text, layout, and other relevant information. + +#### entity\_linking + +```python +def entity_linking(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> EntityLinking +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4120) + +Associates identified entities in the text with specific entries in a knowledge +base or database. + +#### referenceless\_text\_generation\_metric\_default + +```python +def referenceless_text_generation_metric_default( + asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ReferencelessTextGenerationMetricDefault +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4127) + +The Referenceless Text Generation Metric Default is a function designed to +evaluate the quality of generated text without relying on reference texts for +comparison. + +#### fill\_text\_mask + +```python +def fill_text_mask(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> FillTextMask +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4137) + +Completes missing parts of a text based on the context, ideal for content +generation or data augmentation tasks. + +#### subtitling\_translation + +```python +def subtitling_translation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> SubtitlingTranslation +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4144) + +Converts the text of subtitles from one language to another, ensuring context +and cultural nuances are maintained. Essential for global content distribution. + +#### instance\_segmentation + +```python +def instance_segmentation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> InstanceSegmentation +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4151) + +Instance segmentation is a computer vision task that involves detecting and +delineating each distinct object within an image, assigning a unique label and +precise boundary to every individual instance of objects, even if they belong +to the same category. + +#### viseme\_generation + +```python +def viseme_generation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> VisemeGeneration +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4160) + +Viseme Generation is the process of creating visual representations of +phonemes, which are the distinct units of sound in speech, to synchronize lip +movements with spoken words in animations or virtual avatars. + +#### audio\_generation\_metric + +```python +def audio_generation_metric(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> AudioGenerationMetric +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4168) + +The Audio Generation Metric is a quantitative measure used to evaluate the +quality, accuracy, and overall performance of audio generated by artificial +intelligence systems, often considering factors such as fidelity, +intelligibility, and similarity to human-produced audio. + +#### video\_understanding + +```python +def video_understanding(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> VideoUnderstanding +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4177) + +Video Understanding is the process of analyzing and interpreting video content +to extract meaningful information, such as identifying objects, actions, +events, and contextual relationships within the footage. + +#### text\_normalization + +```python +def text_normalization(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextNormalization +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4185) + +Converts unstructured or non-standard textual data into a more readable and +uniform format, dealing with abbreviations, numerals, and other non-standard +words. + +#### asr\_quality\_estimation + +```python +def asr_quality_estimation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> AsrQualityEstimation +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4193) + +ASR Quality Estimation is a process that evaluates the accuracy and reliability +of automatic speech recognition systems by analyzing their performance in +transcribing spoken language into text. + +#### voice\_activity\_detection + +```python +def voice_activity_detection(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> VoiceActivityDetection +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4201) + +Determines when a person is speaking in an audio clip. It's an essential +preprocessing step for other audio-related tasks. + +#### speech\_non\_speech\_classification + +```python +def speech_non_speech_classification( + asset_id: Union[str, asset.Asset], *args, + **kwargs) -> SpeechNonSpeechClassification +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4208) + +Differentiates between speech and non-speech audio segments. Great for editing +software and transcription services to exclude irrelevant audio. + +#### audio\_transcript\_improvement + +```python +def audio_transcript_improvement(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> AudioTranscriptImprovement +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4217) + +Refines and corrects transcriptions generated from audio data, improving +readability and accuracy. + +#### text\_content\_moderation + +```python +def text_content_moderation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextContentModeration +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4224) + +Scans and identifies potentially harmful, offensive, or inappropriate textual +content, ensuring safer user environments. + +#### emotion\_detection + +```python +def emotion_detection(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> EmotionDetection +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4231) + +Identifies human emotions from text or audio, enhancing user experience in +chatbots or customer feedback analysis. + +#### audio\_forced\_alignment + +```python +def audio_forced_alignment(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> AudioForcedAlignment +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4238) + +Synchronizes phonetic and phonological text with the corresponding segments in +an audio file. Useful in linguistic research and detailed transcription tasks. + +#### video\_content\_moderation + +```python +def video_content_moderation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> VideoContentModeration +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4245) + +Automatically reviews video content to detect and possibly remove inappropriate +or harmful material. Essential for user-generated content platforms. + +#### image\_label\_detection + +```python +def image_label_detection(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ImageLabelDetection +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4252) + +Identifies objects, themes, or topics within images, useful for image +categorization, search, and recommendation systems. + +#### video\_forced\_alignment + +```python +def video_forced_alignment(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> VideoForcedAlignment +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4259) + +Aligns the transcription of spoken content in a video with its corresponding +timecodes, facilitating subtitle creation. + +#### text\_generation + +```python +def text_generation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextGeneration +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4266) + +Creates coherent and contextually relevant textual content based on prompts or +certain parameters. Useful for chatbots, content creation, and data +augmentation. + +#### text\_classification + +```python +def text_classification(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextClassification +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4274) + +Categorizes text into predefined groups or topics, facilitating content +organization and targeted actions. + +#### speech\_embedding + +```python +def speech_embedding(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> SpeechEmbedding +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4281) + +Transforms spoken content into a fixed-size vector in a high-dimensional space +that captures the content's essence. Facilitates tasks like speech recognition +and speaker verification. + +#### topic\_classification + +```python +def topic_classification(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TopicClassification +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4289) + +Assigns categories or topics to a piece of text based on its content, +facilitating content organization and retrieval. + +#### translation + +```python +def translation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> Translation +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4296) + +Converts text from one language to another while maintaining the original +message's essence and context. Crucial for global communication. + +#### speech\_recognition + +```python +def speech_recognition(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> SpeechRecognition +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4303) + +Converts spoken language into written text. Useful for transcription services, +voice assistants, and applications requiring voice-to-text capabilities. + +#### subtitling + +```python +def subtitling(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> Subtitling +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4310) + +Generates accurate subtitles for videos, enhancing accessibility for diverse +audiences. + +#### image\_captioning + +```python +def image_captioning(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ImageCaptioning +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4317) + +Image Captioning is a process that involves generating a textual description of +an image, typically using machine learning models to analyze the visual content +and produce coherent and contextually relevant sentences that describe the +objects, actions, and scenes depicted in the image. + +#### audio\_language\_identification + +```python +def audio_language_identification(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> AudioLanguageIdentification +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4326) + +Audio Language Identification is a process that involves analyzing an audio +recording to determine the language being spoken. + +#### video\_embedding + +```python +def video_embedding(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> VideoEmbedding +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4333) + +Video Embedding is a process that transforms video content into a fixed- +dimensional vector representation, capturing essential features and patterns to +facilitate tasks such as retrieval, classification, and recommendation. + +#### asr\_age\_classification + +```python +def asr_age_classification(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> AsrAgeClassification +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4341) + +The ASR Age Classification function is designed to analyze audio recordings of +speech to determine the speaker's age group by leveraging automatic speech +recognition (ASR) technology and machine learning algorithms. + +#### audio\_intent\_detection + +```python +def audio_intent_detection(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> AudioIntentDetection +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4349) + +Audio Intent Detection is a process that involves analyzing audio signals to +identify and interpret the underlying intentions or purposes behind spoken +words, enabling systems to understand and respond appropriately to human +speech. + +#### language\_identification + +```python +def language_identification(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> LanguageIdentification +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4358) + +Detects the language in which a given text is written, aiding in multilingual +platforms or content localization. + +#### ocr + +```python +def ocr(asset_id: Union[str, asset.Asset], *args, **kwargs) -> Ocr +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4365) + +Converts images of typed, handwritten, or printed text into machine-encoded +text. Used in digitizing printed texts for data retrieval. + +#### asr\_gender\_classification + +```python +def asr_gender_classification(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> AsrGenderClassification +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4372) + +The ASR Gender Classification function analyzes audio recordings to determine +and classify the speaker's gender based on their voice characteristics. + +#### language\_identification\_audio + +```python +def language_identification_audio(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> LanguageIdentificationAudio +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4379) + +The Language Identification Audio function analyzes audio input to determine +and identify the language being spoken. + +#### base\_model + +```python +def base_model(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> BaseModel +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4386) + +The Base-Model function serves as a foundational framework designed to provide +essential features and capabilities upon which more specialized or advanced +models can be built and customized. + +#### loglikelihood + +```python +def loglikelihood(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> Loglikelihood +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4394) + +The Log Likelihood function measures the probability of observing the given +data under a specific statistical model by taking the natural logarithm of the +likelihood function, thereby transforming the product of probabilities into a +sum, which simplifies the process of optimization and parameter estimation. + +#### image\_to\_video\_generation + +```python +def image_to_video_generation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ImageToVideoGeneration +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4403) + +The Image To Video Generation function transforms a series of static images +into a cohesive, dynamic video sequence, often incorporating transitions, +effects, and synchronization with audio to create a visually engaging +narrative. + +#### part\_of\_speech\_tagging + +```python +def part_of_speech_tagging(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> PartOfSpeechTagging +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4412) + +Part of Speech Tagging is a natural language processing task that involves +assigning each word in a sentence its corresponding part of speech, such as +noun, verb, adjective, or adverb, based on its role and context within the +sentence. + +#### benchmark\_scoring\_asr + +```python +def benchmark_scoring_asr(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> BenchmarkScoringAsr +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4421) + +Benchmark Scoring ASR is a function that evaluates and compares the performance +of automatic speech recognition systems by analyzing their accuracy, speed, and +other relevant metrics against a standardized set of benchmarks. + +#### visual\_question\_answering + +```python +def visual_question_answering(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> VisualQuestionAnswering +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4429) + +Visual Question Answering (VQA) is a task in artificial intelligence that +involves analyzing an image and providing accurate, contextually relevant +answers to questions posed about the visual content of that image. + +#### document\_information\_extraction + +```python +def document_information_extraction(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> DocumentInformationExtraction +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4437) + +Document Information Extraction is the process of automatically identifying, +extracting, and structuring relevant data from unstructured or semi-structured +documents, such as invoices, receipts, contracts, and forms, to facilitate +easier data management and analysis. + +#### video\_generation + +```python +def video_generation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> VideoGeneration +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4448) + +Produces video content based on specific inputs or datasets. Can be used for +simulations, animations, or even deepfake detection. + +#### multi\_class\_image\_classification + +```python +def multi_class_image_classification( + asset_id: Union[str, asset.Asset], *args, + **kwargs) -> MultiClassImageClassification +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4455) + +Multi Class Image Classification is a machine learning task where an algorithm +is trained to categorize images into one of several predefined classes or +categories based on their visual content. + +#### style\_transfer + +```python +def style_transfer(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> StyleTransfer +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4465) + +Style Transfer is a technique in artificial intelligence that applies the +visual style of one image (such as the brushstrokes of a famous painting) to +the content of another image, effectively blending the artistic elements of the +first image with the subject matter of the second. + +#### multi\_class\_text\_classification + +```python +def multi_class_text_classification(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> MultiClassTextClassification +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4474) + +Multi Class Text Classification is a natural language processing task that +involves categorizing a given text into one of several predefined classes or +categories based on its content. + +#### intent\_classification + +```python +def intent_classification(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> IntentClassification +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4484) + +Intent Classification is a natural language processing task that involves +analyzing and categorizing user text input to determine the underlying purpose +or goal behind the communication, such as booking a flight, asking for weather +information, or setting a reminder. + +#### multi\_label\_text\_classification + +```python +def multi_label_text_classification(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> MultiLabelTextClassification +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4493) + +Multi Label Text Classification is a natural language processing task where a +given text is analyzed and assigned multiple relevant labels or categories from +a predefined set, allowing for the text to belong to more than one category +simultaneously. + +#### text\_reconstruction + +```python +def text_reconstruction(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextReconstruction +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4504) + +Text Reconstruction is a process that involves piecing together fragmented or +incomplete text data to restore it to its original, coherent form. + +#### fact\_checking + +```python +def fact_checking(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> FactChecking +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4511) + +Fact Checking is the process of verifying the accuracy and truthfulness of +information, statements, or claims by cross-referencing with reliable sources +and evidence. + +#### inverse\_text\_normalization + +```python +def inverse_text_normalization(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> InverseTextNormalization +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4519) + +Inverse Text Normalization is the process of converting spoken or written +language in its normalized form, such as numbers, dates, and abbreviations, +back into their original, more complex or detailed textual representations. + +#### text\_to\_audio + +```python +def text_to_audio(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextToAudio +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4527) + +The Text to Audio function converts written text into spoken words, allowing +users to listen to the content instead of reading it. + +#### image\_compression + +```python +def image_compression(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ImageCompression +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4534) + +Reduces the size of image files without significantly compromising their visual +quality. Useful for optimizing storage and improving webpage load times. + +#### multilingual\_speech\_recognition + +```python +def multilingual_speech_recognition(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> MultilingualSpeechRecognition +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4541) + +Multilingual Speech Recognition is a technology that enables the automatic +transcription of spoken language into text across multiple languages, allowing +for seamless communication and understanding in diverse linguistic contexts. + +#### text\_generation\_metric\_default + +```python +def text_generation_metric_default(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextGenerationMetricDefault +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4551) + +The "Text Generation Metric Default" function provides a standard set of +evaluation metrics for assessing the quality and performance of text generation +models. + +#### referenceless\_text\_generation\_metric + +```python +def referenceless_text_generation_metric( + asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ReferencelessTextGenerationMetric +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4559) + +The Referenceless Text Generation Metric is a method for evaluating the quality +of generated text without requiring a reference text for comparison, often +leveraging models or algorithms to assess coherence, relevance, and fluency +based on intrinsic properties of the text itself. + +#### audio\_emotion\_detection + +```python +def audio_emotion_detection(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> AudioEmotionDetection +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4570) + +Audio Emotion Detection is a technology that analyzes vocal characteristics and +patterns in audio recordings to identify and classify the emotional state of +the speaker. + +#### keyword\_spotting + +```python +def keyword_spotting(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> KeywordSpotting +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4578) + +Keyword Spotting is a function that enables the detection and identification of +specific words or phrases within a stream of audio, often used in voice- +activated systems to trigger actions or commands based on recognized keywords. + +#### text\_summarization + +```python +def text_summarization(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextSummarization +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4586) + +Extracts the main points from a larger body of text, producing a concise +summary without losing the primary message. + +#### split\_on\_linebreak + +```python +def split_on_linebreak(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> SplitOnLinebreak +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4593) + +The "Split On Linebreak" function divides a given string into a list of +substrings, using linebreaks (newline characters) as the points of separation. + +#### other\_\_multipurpose\_ + +```python +def other__multipurpose_(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> OtherMultipurpose +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4600) + +The "Other (Multipurpose)" function serves as a versatile category designed to +accommodate a wide range of tasks and activities that do not fit neatly into +predefined classifications, offering flexibility and adaptability for various +needs. + +#### speaker\_diarization\_audio + +```python +def speaker_diarization_audio(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> SpeakerDiarizationAudio +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4609) + +Identifies individual speakers and their respective speech segments within an +audio clip. Ideal for multi-speaker recordings or conference calls. + +#### image\_content\_moderation + +```python +def image_content_moderation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> ImageContentModeration +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4616) + +Detects and filters out inappropriate or harmful images, essential for +platforms with user-generated visual content. + +#### text\_denormalization + +```python +def text_denormalization(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextDenormalization +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4623) + +Converts standardized or normalized text into its original, often more +readable, form. Useful in natural language generation tasks. + +#### speaker\_diarization\_video + +```python +def speaker_diarization_video(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> SpeakerDiarizationVideo +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4630) + +Segments a video based on different speakers, identifying when each individual +speaks. Useful for transcriptions and understanding multi-person conversations. + +#### text\_to\_video\_generation + +```python +def text_to_video_generation(asset_id: Union[str, asset.Asset], *args, + **kwargs) -> TextToVideoGeneration +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/pipeline.py#L4637) + +Text To Video Generation is a process that converts written descriptions or +scripts into dynamic, visual video content using advanced algorithms and +artificial intelligence. + diff --git a/docs/api-reference/python/aixplain/modules/pipeline/response.md b/docs/api-reference/python/aixplain/modules/pipeline/response.md new file mode 100644 index 00000000..1819a65a --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/pipeline/response.md @@ -0,0 +1,169 @@ +--- +sidebar_label: response +title: aixplain.modules.pipeline.response +--- + +### PipelineResponse Objects + +```python +@dataclass +class PipelineResponse() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/response.py#L7) + +A response object for pipeline operations. + +This class encapsulates the response from pipeline operations, including +status, error information, timing data, and any additional fields. + +**Attributes**: + +- `status` _ResponseStatus_ - The status of the pipeline operation. +- `error` _Optional[Dict[str, Any]]_ - Error details if operation failed. +- `elapsed_time` _Optional[float]_ - Time taken to complete the operation. +- `data` _Optional[Text]_ - The main response data. +- `url` _Optional[Text]_ - URL for polling or accessing results. +- `additional_fields` _Dict[str, Any]_ - Any extra fields provided. + +#### \_\_init\_\_ + +```python +def __init__(status: ResponseStatus, + error: Optional[Dict[str, Any]] = None, + elapsed_time: Optional[float] = 0.0, + data: Optional[Text] = None, + url: Optional[Text] = "", + **kwargs) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/response.py#L22) + +Initialize a new PipelineResponse instance. + +**Arguments**: + +- `status` _ResponseStatus_ - The status of the pipeline operation. +- `error` _Optional[Dict[str, Any]], optional_ - Error details if operation + failed. Defaults to None. +- `elapsed_time` _Optional[float], optional_ - Time taken to complete the + operation in seconds. Defaults to 0.0. +- `data` _Optional[Text], optional_ - The main response data. + Defaults to None. +- `url` _Optional[Text], optional_ - URL for polling or accessing results. + Defaults to "". +- `**kwargs` - Additional fields to store in the response. + +#### \_\_getattr\_\_ + +```python +def __getattr__(key: str) -> Any +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/response.py#L52) + +Get an attribute from additional_fields if it exists. + +This method is called when an attribute lookup has not found the +attribute in the usual places (i.e., it is not an instance attribute +nor found through the __mro__ chain). + +**Arguments**: + +- `key` _str_ - The name of the attribute to get. + + +**Returns**: + +- `Any` - The value from additional_fields. + + +**Raises**: + +- `AttributeError` - If the key is not found in additional_fields. + +#### get + +```python +def get(key: str, default: Any = None) -> Any +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/response.py#L73) + +Get an attribute value with a default if not found. + +**Arguments**: + +- `key` _str_ - The name of the attribute to get. +- `default` _Any, optional_ - Value to return if key is not found. + Defaults to None. + + +**Returns**: + +- `Any` - The attribute value or default if not found. + +#### \_\_getitem\_\_ + +```python +def __getitem__(key: str) -> Any +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/response.py#L86) + +Get an attribute value using dictionary-style access. + +This method enables dictionary-style access to attributes +(e.g., response["status"]). + +**Arguments**: + +- `key` _str_ - The name of the attribute to get. + + +**Returns**: + +- `Any` - The attribute value. + + +**Raises**: + +- `AttributeError` - If the key is not found. + +#### \_\_repr\_\_ + +```python +def __repr__() -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/response.py#L103) + +Return a string representation of the PipelineResponse. + +**Returns**: + +- `str` - A string in the format "PipelineResponse(status=X, error=Y, ...)" + containing all non-empty fields. + +#### \_\_contains\_\_ + +```python +def __contains__(key: str) -> bool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/pipeline/response.py#L123) + +Check if an attribute exists using 'in' operator. + +This method enables using the 'in' operator to check for attribute +existence (e.g., "status" in response). + +**Arguments**: + +- `key` _str_ - The name of the attribute to check. + + +**Returns**: + +- `bool` - True if the attribute exists, False otherwise. + diff --git a/docs/api-reference/python/aixplain/modules/team_agent/init.md b/docs/api-reference/python/aixplain/modules/team_agent/init.md new file mode 100644 index 00000000..41a568a0 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/team_agent/init.md @@ -0,0 +1,320 @@ +--- +sidebar_label: team_agent +title: aixplain.modules.team_agent +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: Lucas Pavanelli and Thiago Castro Ferreira +Date: August 15th 2024 +Description: + Team Agent Class + +### InspectorTarget Objects + +```python +class InspectorTarget(str, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/__init__.py#L52) + +Target stages for inspector validation in the team agent pipeline. + +This enumeration defines the stages where inspectors can be applied to +validate and ensure quality of the team agent's operation. + +**Attributes**: + +- `INPUT` - Validates the input data before processing. +- `STEPS` - Validates intermediate steps during processing. +- `OUTPUT` - Validates the final output before returning. + +#### \_\_str\_\_ + +```python +def __str__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/__init__.py#L68) + +Return the string value of the enum member. + +**Returns**: + +- `str` - The string value associated with the enum member. + +### TeamAgent Objects + +```python +class TeamAgent(Model, DeployableMixin[Agent]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/__init__.py#L77) + +Advanced AI system capable of using multiple agents to perform a variety of tasks. + +**Attributes**: + +- `id` _Text_ - ID of the Team Agent +- `name` _Text_ - Name of the Team Agent +- `agents` _List[Agent]_ - List of agents that the Team Agent uses. +- `description` _Text, optional_ - description of the Team Agent. Defaults to "". +- `llm_id` _Text, optional_ - large language model. Defaults to GPT-4o (6646261c6eb563165658bbb1). +- `api_key` _str_ - The TEAM API key used for authentication. +- `supplier` _Text_ - Supplier of the Team Agent. +- `version` _Text_ - Version of the Team Agent. +- `cost` _Dict, optional_ - model price. Defaults to None. +- `use_mentalist` _bool_ - Use Mentalist agent for pre-planning. Defaults to True. +- `name`0 _List[Inspector]_ - List of inspectors that the team agent uses. +- `name`1 _List[InspectorTarget]_ - List of targets where the inspectors are applied. Defaults to [InspectorTarget.STEPS]. + +#### run + +```python +def run( + data: Optional[Union[Dict, Text]] = None, + query: Optional[Text] = None, + session_id: Optional[Text] = None, + history: Optional[List[Dict]] = None, + name: Text = "model_process", + timeout: float = 300, + parameters: Dict = {}, + wait_time: float = 0.5, + content: Optional[Union[Dict[Text, Text], List[Text]]] = None, + max_tokens: int = 2048, + max_iterations: int = 30, + output_format: Optional[OutputFormat] = None, + expected_output: Optional[Union[BaseModel, Text, dict]] = None +) -> AgentResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/__init__.py#L183) + +Runs a team agent call. + +**Arguments**: + +- `data` _Optional[Union[Dict, Text]], optional_ - data to be processed by the team agent. Defaults to None. +- `query` _Optional[Text], optional_ - query to be processed by the team agent. Defaults to None. +- `session_id` _Optional[Text], optional_ - conversation Session ID. Defaults to None. +- `history` _Optional[List[Dict]], optional_ - chat history (in case session ID is None). Defaults to None. +- `name` _Text, optional_ - ID given to a call. Defaults to "model_process". +- `timeout` _float, optional_ - total polling time. Defaults to 300. +- `parameters` _Dict, optional_ - optional parameters to the model. Defaults to "\{}". +- `wait_time` _float, optional_ - wait time in seconds between polling calls. Defaults to 0.5. +- `content` _Union[Dict[Text, Text], List[Text]], optional_ - Content inputs to be processed according to the query. Defaults to None. +- `max_tokens` _int, optional_ - maximum number of tokens which can be generated by the agents. Defaults to 2048. +- `query`0 _int, optional_ - maximum number of iterations between the agents. Defaults to 30. +- `query`1 _OutputFormat, optional_ - response format. If not provided, uses the format set during initialization. +- `query`2 _Union[BaseModel, Text, dict], optional_ - expected output. Defaults to None. + +**Returns**: + +- `query`3 - parsed output from model + +#### run\_async + +```python +def run_async( + data: Optional[Union[Dict, Text]] = None, + query: Optional[Text] = None, + session_id: Optional[Text] = None, + history: Optional[List[Dict]] = None, + name: Text = "model_process", + parameters: Dict = {}, + content: Optional[Union[Dict[Text, Text], List[Text]]] = None, + max_tokens: int = 2048, + max_iterations: int = 30, + output_format: Optional[OutputFormat] = None, + expected_output: Optional[Union[BaseModel, Text, dict]] = None +) -> AgentResponse +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/__init__.py#L273) + +Runs asynchronously a Team Agent call. + +**Arguments**: + +- `data` _Optional[Union[Dict, Text]], optional_ - data to be processed by the Team Agent. Defaults to None. +- `query` _Optional[Text], optional_ - query to be processed by the Team Agent. Defaults to None. +- `session_id` _Optional[Text], optional_ - conversation Session ID. Defaults to None. +- `history` _Optional[List[Dict]], optional_ - chat history (in case session ID is None). Defaults to None. +- `name` _Text, optional_ - ID given to a call. Defaults to "model_process". +- `parameters` _Dict, optional_ - optional parameters to the model. Defaults to "\{}". +- `content` _Union[Dict[Text, Text], List[Text]], optional_ - Content inputs to be processed according to the query. Defaults to None. +- `max_tokens` _int, optional_ - maximum number of tokens which can be generated by the agents. Defaults to 2048. +- `max_iterations` _int, optional_ - maximum number of iterations between the agents. Defaults to 30. +- `output_format` _OutputFormat, optional_ - response format. If not provided, uses the format set during initialization. +- `query`0 _Union[BaseModel, Text, dict], optional_ - expected output. Defaults to None. + +**Returns**: + +- `query`1 - polling URL in response + +#### delete + +```python +def delete() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/__init__.py#L402) + +Delete Corpus service + +#### to\_dict + +```python +def to_dict() -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/__init__.py#L461) + +Convert the TeamAgent instance to a dictionary representation. + +This method serializes the TeamAgent and all its components (agents, +inspectors, LLMs, etc.) into a dictionary format suitable for storage +or transmission. + +**Returns**: + +- `Dict` - A dictionary containing: + - id (str): The team agent's ID + - name (str): The team agent's name + - agents (List[Dict]): Serialized list of agents + - links (List): Empty list (reserved for future use) + - description (str): The team agent's description + - llmId (str): ID of the main language model + - supervisorId (str): ID of the supervisor language model + - plannerId (str): ID of the planner model (if use_mentalist) + - inspectors (List[Dict]): Serialized list of inspectors + - inspectorTargets (List[str]): List of inspector target stages + - supplier (str): The supplier code + - version (str): The version number + - status (str): The current status + - role (str): The team agent's instructions + +#### from\_dict + +```python +@classmethod +def from_dict(cls, data: Dict) -> "TeamAgent" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/__init__.py#L509) + +Create a TeamAgent instance from a dictionary representation. + +**Arguments**: + +- `data` - Dictionary containing TeamAgent parameters + + +**Returns**: + + TeamAgent instance + +#### validate + +```python +def validate(raise_exception: bool = False) -> bool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/__init__.py#L630) + +Validate the TeamAgent configuration. + +This method checks the validity of the TeamAgent's configuration, +including name format, LLM compatibility, and agent validity. + +**Arguments**: + +- `raise_exception` _bool, optional_ - If True, raises exceptions for + validation failures. If False, logs warnings. Defaults to False. + + +**Returns**: + +- `bool` - True if validation succeeds, False otherwise. + + +**Raises**: + +- `Exception` - If raise_exception is True and validation fails, with + details about the specific validation error. + + +**Notes**: + + - The team agent cannot be run until all validation issues are fixed + - Name must contain only alphanumeric chars, spaces, hyphens, brackets + - LLM must be a text generation model + - All agents must pass their own validation + +#### update + +```python +def update() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/__init__.py#L666) + +Update the TeamAgent in the backend. + +This method validates and updates the TeamAgent's configuration in the +backend system. It is deprecated in favor of the save() method. + +**Raises**: + +- `Exception` - If validation fails or if the update request fails. + Specific error messages will indicate: + - Validation failures with details + - HTTP errors with status codes + - General update errors requiring admin attention + + +**Notes**: + + - This method is deprecated, use save() instead + - Performs validation before attempting update + - Requires valid team API key for authentication + - Returns a new TeamAgent instance if successful + +#### save + +```python +def save() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/__init__.py#L718) + +Save the Agent. + +#### \_\_repr\_\_ + +```python +def __repr__() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/__init__.py#L722) + +Return a string representation of the TeamAgent. + +**Returns**: + +- `str` - A string in the format "TeamAgent: <name> (id=<id>)". + diff --git a/docs/api-reference/python/aixplain/modules/team_agent/inspector.md b/docs/api-reference/python/aixplain/modules/team_agent/inspector.md new file mode 100644 index 00000000..f158f243 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/team_agent/inspector.md @@ -0,0 +1,156 @@ +--- +sidebar_label: inspector +title: aixplain.modules.team_agent.inspector +--- + +Pre-defined agent for inspecting the data flow within a team agent. + +WARNING: This feature is currently in private beta. + +Example usage: + +inspector = Inspector( + name="my_inspector", + model_id="my_model", + model_config=\{"prompt": "Check if the data is safe to use."}, + policy=InspectorPolicy.ADAPTIVE +) + +team = TeamAgent( + name="team" + agents=agents, + description="team description", + llm_id="xyz", + use_mentalist=True, + inspectors=[inspector], +) + +#### AUTO\_DEFAULT\_MODEL\_ID + +GPT-4.1 Nano + +### InspectorAuto Objects + +```python +class InspectorAuto(str, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/inspector.py#L35) + +A list of keywords for inspectors configured automatically in the backend. + +#### get\_name + +```python +def get_name() -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/inspector.py#L40) + +Get the standardized name for this inspector type. + +This method generates a consistent name for the inspector by prefixing +the enum value with "inspector_". + +**Returns**: + +- `Text` - The inspector name in the format "inspector_<type>". + +### InspectorPolicy Objects + +```python +class InspectorPolicy(str, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/inspector.py#L52) + +Which action to take if the inspector gives negative feedback. + +#### WARN + +log only, continue execution + +#### ABORT + +stop execution + +#### ADAPTIVE + +adjust execution according to feedback + +### Inspector Objects + +```python +class Inspector(ModelWithParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/inspector.py#L60) + +Pre-defined agent for inspecting the data flow within a team agent. + +The model should be onboarded before using it as an inspector. + +**Attributes**: + +- `name` - The name of the inspector. +- `model_id` - The ID of the model to wrap. +- `model_params` - The configuration for the model. +- `policy` - The policy for the inspector. Default is ADAPTIVE. + +#### \_\_init\_\_ + +```python +def __init__(*args, **kwargs) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/inspector.py#L77) + +Initialize an Inspector instance. + +This method initializes an inspector with either a custom model or an +automatic configuration. If auto is specified, it uses the default +auto model ID. + +**Arguments**: + +- `*args` - Variable length argument list passed to parent class. +- `**kwargs` - Arbitrary keyword arguments. Supported keys: + - name (Text): The inspector's name + - model_id (Text): The model ID to use + - model_params (Dict, optional): Model configuration + - auto (InspectorAuto, optional): Auto configuration type + - policy (InspectorPolicy, optional): Inspector policy + + +**Notes**: + + If auto is specified in kwargs, model_id is automatically set to + AUTO_DEFAULT_MODEL_ID. + +#### validate\_name + +```python +@field_validator("name") +def validate_name(cls, v: Text) -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/team_agent/inspector.py#L102) + +Validate the inspector name field. + +This validator ensures that the inspector's name is not empty. + +**Arguments**: + +- `v` _Text_ - The name value to validate. + + +**Returns**: + +- `Text` - The validated name value. + + +**Raises**: + +- `ValueError` - If the name is an empty string. + diff --git a/docs/api-reference/python/aixplain/modules/wallet.md b/docs/api-reference/python/aixplain/modules/wallet.md new file mode 100644 index 00000000..254ad6d6 --- /dev/null +++ b/docs/api-reference/python/aixplain/modules/wallet.md @@ -0,0 +1,61 @@ +--- +sidebar_label: wallet +title: aixplain.modules.wallet +--- + +#### \_\_author\_\_ + +Copyright 2024 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Author: aiXplain Team +Date: August 20th 2024 +Description: + Wallet Class + +### Wallet Objects + +```python +class Wallet() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/wallet.py#L25) + +A class representing a wallet for managing credit balances. + +This class provides functionality for managing credit balances in a wallet, +including total, reserved, and available balances. It is used to track and +manage credit resources in the aiXplain platform. + +**Attributes**: + +- `total_balance` _float_ - Total credit balance in the wallet. +- `reserved_balance` _float_ - Reserved credit balance in the wallet. +- `available_balance` _float_ - Available balance (total - reserved). + +#### \_\_init\_\_ + +```python +def __init__(total_balance: float, reserved_balance: float) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/modules/wallet.py#L37) + +Initialize a new Wallet instance. + +**Arguments**: + +- `total_balance` _float_ - Total credit balance in the wallet. +- `reserved_balance` _float_ - Reserved credit balance in the wallet. + diff --git a/docs/api-reference/python/aixplain/processes/data_onboarding/init.md b/docs/api-reference/python/aixplain/processes/data_onboarding/init.md new file mode 100644 index 00000000..a957d8dc --- /dev/null +++ b/docs/api-reference/python/aixplain/processes/data_onboarding/init.md @@ -0,0 +1,6 @@ +--- +draft: true +sidebar_label: data_onboarding +title: aixplain.processes.data_onboarding +--- + diff --git a/docs/api-reference/python/aixplain/processes/data_onboarding/onboard_functions.md b/docs/api-reference/python/aixplain/processes/data_onboarding/onboard_functions.md new file mode 100644 index 00000000..061e1722 --- /dev/null +++ b/docs/api-reference/python/aixplain/processes/data_onboarding/onboard_functions.md @@ -0,0 +1,299 @@ +--- +sidebar_label: onboard_functions +title: aixplain.processes.data_onboarding.onboard_functions +--- + +#### get\_paths + +```python +def get_paths(input_paths: List[Union[str, Path]]) -> List[Path] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/processes/data_onboarding/onboard_functions.py#L40) + +Recursively collect all supported local file paths from the given input paths. + +This function traverses through the provided paths, which can be files or directories, +and collects paths to all supported files (currently only CSV files). It also performs +size validation to ensure files don't exceed 1GB. + +**Arguments**: + +- `input_paths` _List[Union[str, Path]]_ - List of input paths. Can include both + individual file paths and directory paths. + + +**Returns**: + +- `List[Path]` - List of validated local file paths that are supported. + + +**Raises**: + +- `AssertionError` - If any CSV file exceeds 1GB in size. +- `Warning` - If a file has an unsupported extension. + +#### process\_data\_files + +```python +def process_data_files( + data_asset_name: str, + metadata: MetaData, + paths: List, + folder: Optional[Union[str, Path]] = None +) -> Tuple[List[File], int, int, int, int] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/processes/data_onboarding/onboard_functions.py#L83) + +Process data files based on their type and prepare them for upload to S3. + +This function handles different types of data files (audio, image, text, etc.) +by delegating to appropriate processing modules. It compresses the files if needed +and prepares them for upload to S3. + +**Arguments**: + +- `data_asset_name` _str_ - Name of the data asset being processed. +- `metadata` _MetaData_ - Metadata object containing type and subtype information + for the data being processed. +- `paths` _List_ - List of paths to local files that need processing. +- `folder` _Optional[Union[str, Path]], optional_ - Local folder to save processed + files before uploading to S3. If None, uses data_asset_name. Defaults to None. + + +**Returns**: + + Tuple[List[File], int, int, int, int]: A tuple containing: + - List[File]: List of processed file objects ready for S3 upload + - int: Index of the data column + - int: Index of the start column (for intervals) + - int: Index of the end column (for intervals) + - int: Total number of rows processed + +#### build\_payload\_data + +```python +def build_payload_data(data: Data) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/processes/data_onboarding/onboard_functions.py#L129) + +Build a payload dictionary for data onboarding to the core engine. + +This function creates a standardized payload structure for onboarding data +to the core engine. It includes data properties, file information, and metadata +such as languages and column mappings. + +**Arguments**: + +- `data` _Data_ - Data object containing information about the data to be onboarded, + including name, type, files, and language information. + + +**Returns**: + +- `Dict` - A dictionary containing the formatted payload with the following key fields: + - name: Name of the data + - dataColumn: Column identifier for the data + - dataType: Type of the data + - dataSubtype: Subtype of the data + - batches: List of file information with paths and order + - tags: List of descriptive tags + - metaData: Additional metadata including languages + Additional fields may be added for interval data (start/end columns). + +#### build\_payload\_corpus + +```python +def build_payload_corpus(corpus: Corpus, ref_data: List[Text], + error_handler: ErrorHandler) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/processes/data_onboarding/onboard_functions.py#L174) + +Build a payload dictionary for corpus onboarding to the core engine. + +This function creates a standardized payload structure for onboarding a corpus, +including all its associated data, metadata, and configuration settings. + +**Arguments**: + +- `corpus` _Corpus_ - Corpus object containing the data collection to be onboarded, + including name, description, functions, and associated data. +- `ref_data` _List[Text]_ - List of referenced data IDs that this corpus depends on + or is related to. +- `error_handler` _ErrorHandler_ - Configuration for how to handle rows that fail + during the onboarding process. + + +**Returns**: + +- `Dict` - A dictionary containing the formatted payload with the following key fields: + - name: Name of the corpus + - description: Description of the corpus + - suggestedFunctions: List of suggested AI functions + - onboardingErrorsPolicy: Error handling policy + - tags: List of descriptive tags + - pricing: Pricing configuration + - privacy: Privacy settings + - license: License information + - refData: Referenced data IDs + - data: List of data payloads for each data component + +#### build\_payload\_dataset + +```python +def build_payload_dataset(dataset: Dataset, input_ref_data: Dict[Text, Any], + output_ref_data: Dict[Text, List[Any]], + hypotheses_ref_data: Dict[Text, Any], + meta_ref_data: Dict[Text, Any], tags: List[Text], + error_handler: ErrorHandler) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/processes/data_onboarding/onboard_functions.py#L220) + +Build a payload dictionary for dataset onboarding to the core engine. + +This function creates a comprehensive payload structure for onboarding a dataset, +including all its components: input data, output data, hypotheses, and metadata. +It handles both new data and references to existing data. + +**Arguments**: + +- `dataset` _Dataset_ - Dataset object to be onboarded, containing all the data + components and configuration. +- `input_ref_data` _Dict[Text, Any]_ - Dictionary mapping input names to existing + data IDs in the system. +- `output_ref_data` _Dict[Text, List[Any]]_ - Dictionary mapping output names to + lists of existing data IDs for multi-reference outputs. +- `hypotheses_ref_data` _Dict[Text, Any]_ - Dictionary mapping hypothesis names to + existing data IDs for model outputs or predictions. +- `meta_ref_data` _Dict[Text, Any]_ - Dictionary mapping metadata names to existing + metadata IDs in the system. +- `tags` _List[Text]_ - List of descriptive tags for the dataset. +- `error_handler` _ErrorHandler_ - Configuration for how to handle rows that fail + during the onboarding process. + + +**Returns**: + +- `Dict` - A dictionary containing the formatted payload with the following sections: + - Basic information (name, description, function, etc.) + - Configuration (error handling, privacy, license) + - Input data section with both new and referenced inputs + - Output data section with both new and referenced outputs + - Hypotheses section with both new and referenced hypotheses + - Metadata section with both new and referenced metadata + +#### create\_data\_asset + +```python +def create_data_asset(payload: Dict, + data_asset_type: Text = "corpus", + api_key: Optional[Text] = None) -> Dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/processes/data_onboarding/onboard_functions.py#L355) + +Create a new data asset (corpus or dataset) in the core engine. + +This function sends the onboarding request to the core engine and handles the response. +It supports both corpus and dataset creation with proper authentication. + +**Arguments**: + +- `payload` _Dict_ - The complete payload for the data asset, containing all necessary + information for onboarding (structure depends on data_asset_type). +- `data_asset_type` _Text, optional_ - Type of data asset to create. Must be either + "corpus" or "dataset". Defaults to "corpus". +- `api_key` _Optional[Text], optional_ - Team API key for authentication. If None, + uses the default key from config. Defaults to None. + + +**Returns**: + +- `Dict` - A dictionary containing the onboarding status with the following fields: + - success (bool): Whether the operation was successful + - asset_id (str): ID of the created asset (if successful) + - status (str): Current status of the asset (if successful) + - error (str): Error message (if not successful) + + +**Notes**: + + The function handles both successful and failed responses, providing appropriate + error messages in case of failure. + +#### is\_data + +```python +def is_data(data_id: Text) -> bool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/processes/data_onboarding/onboard_functions.py#L413) + +Check if a data object exists in the system by its ID. + +This function makes an API call to verify the existence of a data object +in the system. It's typically used to validate references before creating +new assets that depend on existing data. + +**Arguments**: + +- `data_id` _Text_ - The ID of the data object to check. + + +**Returns**: + +- `bool` - True if the data exists and is accessible, False otherwise. + Returns False in case of API errors or if the data is not found. + + +**Notes**: + + The function handles API errors gracefully, returning False instead + of raising exceptions. + +#### split\_data + +```python +def split_data(paths: List, split_rate: List[float], + split_labels: List[Text]) -> MetaData +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/processes/data_onboarding/onboard_functions.py#L446) + +Split data files into partitions based on specified rates and labels. + +This function adds a new column to CSV files to indicate the split assignment +for each row. It randomly assigns rows to splits based on the provided rates. +The function tries to find an unused column name for the split information. + +**Arguments**: + +- `paths` _List_ - List of paths to CSV files that need to be split. +- `split_rate` _List[float]_ - List of proportions for each split. Should sum to 1.0. + For example, [0.8, 0.1, 0.1] for train/dev/test split. +- `split_labels` _List[Text]_ - List of labels corresponding to each split rate. + For example, ["train", "dev", "test"]. + + +**Returns**: + +- `MetaData` - A metadata object for the new split column with: + - name: The generated column name for the split + - dtype: Set to DataType.LABEL + - dsubtype: Set to DataSubtype.SPLIT + - storage_type: Set to StorageType.TEXT + + +**Raises**: + +- `Exception` - If no available column name is found or if file operations fail. + + +**Notes**: + + The function modifies the input CSV files in place, adding the new split column. + diff --git a/docs/api-reference/python/aixplain/processes/data_onboarding/process_media_files.md b/docs/api-reference/python/aixplain/processes/data_onboarding/process_media_files.md new file mode 100644 index 00000000..70d5f5f1 --- /dev/null +++ b/docs/api-reference/python/aixplain/processes/data_onboarding/process_media_files.md @@ -0,0 +1,83 @@ +--- +sidebar_label: process_media_files +title: aixplain.processes.data_onboarding.process_media_files +--- + +#### compress\_folder + +```python +def compress_folder(folder_path: str) -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/processes/data_onboarding/process_media_files.py#L25) + +Compress a folder into a gzipped tar archive. + +This function takes a folder and creates a compressed tar archive (.tgz) +containing all files in the folder. The archive is created in the same +directory as the input folder. + +**Arguments**: + +- `folder_path` _str_ - Path to the folder to be compressed. + + +**Returns**: + +- `str` - Path to the created .tgz archive file. + +#### run + +```python +def run(metadata: MetaData, + paths: List, + folder: Path, + batch_size: int = 100) -> Tuple[List[File], int, int, int, int] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/processes/data_onboarding/process_media_files.py#L44) + +Process media files and prepare them for upload to S3 with batch processing. + +This function handles the processing and uploading of media files (audio, image, etc.) +to S3. It supports both local files and public URLs, processes them in batches, +and creates index files to track the media locations and any interval information. + +The process works as follows: +1. For each media file in the input paths: +- If it's a public URL: Add the URL to an index CSV file +- If it's a local file: Copy to a temporary folder and add path to index +2. After every batch_size files: +- For local files: Compress the folder into .tgz and upload to S3 +- Create and upload an index CSV file with paths and metadata +- Reset for the next batch + +**Arguments**: + +- `metadata` _MetaData_ - Metadata object containing information about the media type, + storage type, and column mappings. +- `paths` _List_ - List of paths to CSV files containing media information. +- `folder` _Path_ - Local folder path where temporary files and compressed archives + will be stored during processing. +- `batch_size` _int, optional_ - Number of media files to process in each batch. + Defaults to 100. + + +**Returns**: + + Tuple[List[File], int, int, int, int]: A tuple containing: + - List[File]: List of File objects pointing to uploaded index files in S3 + - int: Index of the data column in the index CSV + - int: Index of the start column for intervals (-1 if not used) + - int: Index of the end column for intervals (-1 if not used) + - int: Total number of media files processed + + +**Raises**: + +- `Exception` - If: + - Input files are not found + - Required columns are missing + - File size limits are exceeded (50MB for audio, 25MB for others) + - Invalid interval configurations are detected + diff --git a/docs/api-reference/python/aixplain/processes/data_onboarding/process_text_files.md b/docs/api-reference/python/aixplain/processes/data_onboarding/process_text_files.md new file mode 100644 index 00000000..611a326b --- /dev/null +++ b/docs/api-reference/python/aixplain/processes/data_onboarding/process_text_files.md @@ -0,0 +1,98 @@ +--- +sidebar_label: process_text_files +title: aixplain.processes.data_onboarding.process_text_files +--- + +#### process\_text + +```python +def process_text(content: str, storage_type: StorageType) -> Text +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/processes/data_onboarding/process_text_files.py#L18) + +Process text content based on its storage type and location. + +This function handles different types of text content: +- Local files: Reads the file content (with size validation) +- URLs: Marks them for non-download if they're public links +- Direct text: Uses the content as-is + +**Arguments**: + +- `content` _str_ - The text content to process. Can be: + - A path to a local file + - A URL pointing to text content + - The actual text content +- `storage_type` _StorageType_ - The type of storage for the content: + - StorageType.FILE for local files + - StorageType.TEXT for direct text content + - Other storage types for different handling + + +**Returns**: + +- `Text` - The processed text content. URLs may be prefixed with + "DONOTDOWNLOAD" if they should not be downloaded. + + +**Raises**: + +- `AssertionError` - If a local text file exceeds 25MB in size. +- `IOError` - If there are issues reading a local file. + +#### run + +```python +def run(metadata: MetaData, + paths: List, + folder: Path, + batch_size: int = 1000) -> Tuple[List[File], int, int] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/processes/data_onboarding/process_text_files.py#L65) + +Process text files in batches and upload them to S3 with index tracking. + +This function processes text files (either local or from URLs) in batches, +creating compressed CSV index files that track the text content and their +positions. The index files are then uploaded to S3. + +The process works as follows: +1. For each input CSV file: +- Read the specified column containing text content/paths +- Process each text entry (read files, handle URLs) +- Add processed text to the current batch +2. After every batch_size entries: +- Create a new index CSV with the processed texts +- Add row indices for tracking +- Compress and upload the index to S3 +- Start a new batch + +**Arguments**: + +- `metadata` _MetaData_ - Metadata object containing information about the text data, + including column names and storage type configuration. +- `paths` _List_ - List of paths to CSV files containing the text data or + references to text content. +- `folder` _Path_ - Local folder path where the generated index files will be + temporarily stored before upload. +- `batch_size` _int, optional_ - Number of text entries to process in each batch. + Defaults to 1000. + + +**Returns**: + + Tuple[List[File], int, int]: A tuple containing: + - List[File]: List of File objects pointing to uploaded index files in S3 + - int: Index of the data column in the index CSV files + - int: Total number of text entries processed + + +**Raises**: + +- `Exception` - If: + - Input CSV files are not found + - Required columns are missing in input files + - Text processing fails (e.g., file size limit exceeded) + diff --git a/docs/api-reference/python/aixplain/processes/init.md b/docs/api-reference/python/aixplain/processes/init.md new file mode 100644 index 00000000..55a1d966 --- /dev/null +++ b/docs/api-reference/python/aixplain/processes/init.md @@ -0,0 +1,6 @@ +--- +draft: true +sidebar_label: processes +title: aixplain.processes +--- + diff --git a/docs/api-reference/python/aixplain/utils/asset_cache.md b/docs/api-reference/python/aixplain/utils/asset_cache.md new file mode 100644 index 00000000..ea25d996 --- /dev/null +++ b/docs/api-reference/python/aixplain/utils/asset_cache.md @@ -0,0 +1,277 @@ +--- +sidebar_label: asset_cache +title: aixplain.utils.asset_cache +--- + +### Store Objects + +```python +@dataclass +class Store(Generic[T]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/asset_cache.py#L24) + +A generic data store for cached assets with expiration time. + +This class serves as a container for cached data and its expiration timestamp. +It is used internally by AssetCache to store the cached assets. + +**Attributes**: + +- `data` _Dict[str, T]_ - Dictionary mapping asset IDs to their cached instances. +- `expiry` _int_ - Unix timestamp when the cached data expires. + +### AssetCache Objects + +```python +class AssetCache(Generic[T]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/asset_cache.py#L38) + +A modular caching system for aiXplain assets with file-based persistence. + +This class provides a generic caching mechanism for different types of assets +(Models, Pipelines, Agents, etc.) with automatic serialization, expiration, +and thread-safe file persistence. + +The cache uses JSON files for storage and implements file locking to ensure +thread safety. It also supports automatic cache invalidation based on +expiration time. + +**Attributes**: + +- `cls` _Type[T]_ - The class type of assets to be cached. +- `cache_file` _str_ - Path to the JSON file storing the cached data. +- `lock_file` _str_ - Path to the lock file for thread-safe operations. +- `store` _Store[T]_ - The in-memory store containing cached data and expiry. + + +**Notes**: + + The cached assets must be serializable to JSON and should implement + either a to_dict() method or have a standard __dict__ attribute. + +#### \_\_init\_\_ + +```python +def __init__(cls: Type[T], cache_filename: Optional[str] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/asset_cache.py#L60) + +Initialize a new AssetCache instance. + +**Arguments**: + +- `cls` _Type[T]_ - The class type of assets to be cached. Must be + serializable to JSON. +- `cache_filename` _Optional[str], optional_ - Base name for the cache file. + If None, uses lowercase class name. Defaults to None. + +#### compute\_expiry + +```python +def compute_expiry() -> int +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/asset_cache.py#L86) + +Calculate the expiration timestamp for cached data. + +Uses CACHE_EXPIRY_TIME environment variable if set, otherwise falls back +to the default CACHE_DURATION. The expiry is calculated as current time +plus the duration. + +**Returns**: + +- `int` - Unix timestamp when the cache will expire. + + +**Notes**: + + If CACHE_EXPIRY_TIME is invalid, it will be removed from environment + variables and the default duration will be used. + +#### invalidate + +```python +def invalidate() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/asset_cache.py#L113) + +Clear the cache and remove cache files. + +This method: +1. Resets the in-memory store with empty data and new expiry +2. Deletes the cache file if it exists +3. Deletes the lock file if it exists + +#### load + +```python +def load() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/asset_cache.py#L128) + +Load cached data from the cache file. + +This method reads the cache file (if it exists) and loads the data into +the in-memory store. It performs the following: +1. Checks if cache file exists, if not, invalidates cache +2. Uses file locking to ensure thread-safe reading +3. Deserializes JSON data and converts to appropriate asset instances +4. Checks expiration time and invalidates if expired +5. Handles any errors by invalidating the cache + +**Notes**: + + If any errors occur during loading (file not found, invalid JSON, + deserialization errors), the cache will be invalidated. + +#### save + +```python +def save() -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/asset_cache.py#L174) + +Save the current cache state to the cache file. + +This method serializes the current cache state to JSON and writes it +to the cache file. It performs the following: +1. Creates the cache directory if it doesn't exist +2. Uses file locking to ensure thread-safe writing +3. Serializes each cached asset to a JSON-compatible format +4. Writes the serialized data and expiry time to the cache file + +**Notes**: + + If serialization fails for any asset, that asset will be skipped + and an error will be logged, but the save operation will continue + for other assets. + +#### get + +```python +def get(asset_id: str) -> Optional[T] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/asset_cache.py#L207) + +Retrieve a cached asset by its ID. + +**Arguments**: + +- `asset_id` _str_ - The unique identifier of the asset to retrieve. + + +**Returns**: + +- `Optional[T]` - The cached asset instance if found, None otherwise. + +#### add + +```python +def add(asset: T) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/asset_cache.py#L218) + +Add a single asset to the cache. + +**Arguments**: + +- `asset` _T_ - The asset instance to cache. Must have an 'id' attribute + and be serializable to JSON. + + +**Notes**: + + This method automatically saves the updated cache to disk after + adding the asset. + +#### add\_list + +```python +def add_list(assets: List[T]) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/asset_cache.py#L232) + +Add multiple assets to the cache at once. + +This method replaces all existing cached assets with the new list. + +**Arguments**: + +- `assets` _List[T]_ - List of asset instances to cache. Each asset must + have an 'id' attribute and be serializable to JSON. + + +**Notes**: + + This method automatically saves the updated cache to disk after + adding the assets. + +#### get\_all + +```python +def get_all() -> List[T] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/asset_cache.py#L248) + +Retrieve all cached assets. + +**Returns**: + +- `List[T]` - List of all cached asset instances. Returns an empty list + if the cache is empty. + +#### has\_valid\_cache + +```python +def has_valid_cache() -> bool +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/asset_cache.py#L257) + +Check if the cache is valid and not expired. + +**Returns**: + +- `bool` - True if the cache has not expired and contains data, + False otherwise. + +#### serialize + +```python +def serialize(obj: Any) -> Any +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/asset_cache.py#L266) + +Convert a Python object into a JSON-serializable format. + +This function handles various Python types and converts them to formats +that can be serialized to JSON. It supports: +- Basic types (str, int, float, bool, None) +- Collections (list, tuple, set, dict) +- Objects with to_dict() method +- Objects with __dict__ attribute +- Other objects (converted to string) + +**Arguments**: + +- `obj` _Any_ - The Python object to serialize. + + +**Returns**: + +- `Any` - A JSON-serializable version of the input object. + diff --git a/docs/api-reference/python/aixplain/utils/cache_utils.md b/docs/api-reference/python/aixplain/utils/cache_utils.md new file mode 100644 index 00000000..f2c0e276 --- /dev/null +++ b/docs/api-reference/python/aixplain/utils/cache_utils.md @@ -0,0 +1,79 @@ +--- +sidebar_label: cache_utils +title: aixplain.utils.cache_utils +--- + +#### get\_cache\_expiry + +```python +def get_cache_expiry() -> int +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/cache_utils.py#L13) + +Get the cache expiration duration in seconds. + +Retrieves the cache expiration duration from the CACHE_EXPIRY_TIME +environment variable. If not set, falls back to the default CACHE_DURATION. + +**Returns**: + +- `int` - The cache expiration duration in seconds. + +#### save\_to\_cache + +```python +def save_to_cache(cache_file: str, data: dict, lock_file: str) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/cache_utils.py#L25) + +Save data to a cache file with thread-safe file locking. + +This function saves the provided data to a JSON cache file along with a +timestamp. It uses file locking to ensure thread safety during writing. + +**Arguments**: + +- `cache_file` _str_ - Path to the cache file where data will be saved. +- `data` _dict_ - The data to be cached. Must be JSON-serializable. +- `lock_file` _str_ - Path to the lock file used for thread safety. + + +**Notes**: + + - Creates the cache directory if it doesn't exist + - Logs an error if saving fails but doesn't raise an exception + - The data is saved with a timestamp for expiration checking + +#### load\_from\_cache + +```python +def load_from_cache(cache_file: str, lock_file: str) -> dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/cache_utils.py#L50) + +Load data from a cache file with expiration checking. + +This function loads data from a JSON cache file if it exists and hasn't +expired. It uses file locking to ensure thread safety during reading. + +**Arguments**: + +- `cache_file` _str_ - Path to the cache file to load data from. +- `lock_file` _str_ - Path to the lock file used for thread safety. + + +**Returns**: + +- `dict` - The cached data if the cache exists and hasn't expired, + None otherwise. + + +**Notes**: + + - Returns None if the cache file doesn't exist + - Returns None if the cached data has expired based on CACHE_EXPIRY_TIME + - Uses thread-safe file locking for reading + diff --git a/docs/api-reference/python/aixplain/utils/config.md b/docs/api-reference/python/aixplain/utils/config.md new file mode 100644 index 00000000..25dc82ae --- /dev/null +++ b/docs/api-reference/python/aixplain/utils/config.md @@ -0,0 +1,19 @@ +--- +sidebar_label: config +title: aixplain.utils.config +--- + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + diff --git a/docs/api-reference/python/aixplain/utils/convert_datatype_utils.md b/docs/api-reference/python/aixplain/utils/convert_datatype_utils.md new file mode 100644 index 00000000..db57c54d --- /dev/null +++ b/docs/api-reference/python/aixplain/utils/convert_datatype_utils.md @@ -0,0 +1,43 @@ +--- +sidebar_label: convert_datatype_utils +title: aixplain.utils.convert_datatype_utils +--- + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +#### dict\_to\_metadata + +```python +def dict_to_metadata(metadatas: List[Union[Dict, MetaData]]) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/convert_datatype_utils.py#L21) + +Convert all the Dicts to MetaData + +**Arguments**: + +- `metadatas` _List[Union[Dict, MetaData]], optional_ - metadata of metadata information of the dataset. + + +**Returns**: + + None + + +**Raises**: + +- `TypeError` - If one or more elements in the metadata_schema are not well-structured + diff --git a/docs/api-reference/python/aixplain/utils/file_utils.md b/docs/api-reference/python/aixplain/utils/file_utils.md new file mode 100644 index 00000000..97a251bc --- /dev/null +++ b/docs/api-reference/python/aixplain/utils/file_utils.md @@ -0,0 +1,198 @@ +--- +sidebar_label: file_utils +title: aixplain.utils.file_utils +--- + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +#### save\_file + +```python +def save_file( + download_url: Text, + download_file_path: Optional[Union[str, + Path]] = None) -> Union[str, Path] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/file_utils.py#L32) + +Download and save a file from a given URL. + +This function downloads a file from the specified URL and saves it either +to a specified path or to a generated path in the 'aiXplain' directory. + +**Arguments**: + +- `download_url` _Text_ - URL of the file to download. +- `download_file_path` _Optional[Union[str, Path]], optional_ - Path where the + downloaded file should be saved. If None, generates a folder 'aiXplain' + in the current working directory and saves the file there with a UUID + name. Defaults to None. + + +**Returns**: + + Union[str, Path]: Path where the file was downloaded. + + +**Notes**: + + If download_file_path is None, the file will be saved with a UUID name + and the original file extension in the 'aiXplain' directory. + +#### download\_data + +```python +def download_data(url_link: str, local_filename: Optional[str] = None) -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/file_utils.py#L64) + +Download a file from a URL with streaming support. + +This function downloads a file from the specified URL using streaming to +handle large files efficiently. The file is downloaded in chunks to +minimize memory usage. + +**Arguments**: + +- `url_link` _str_ - URL of the file to download. +- `local_filename` _Optional[str], optional_ - Local path where the file + should be saved. If None, uses the last part of the URL as the + filename. Defaults to None. + + +**Returns**: + +- `str` - Path to the downloaded file. + + +**Raises**: + +- `requests.exceptions.RequestException` - If the download fails or the + server returns an error status. + +#### upload\_data + +```python +def upload_data(file_name: Union[Text, Path], + tags: Optional[List[Text]] = None, + license: Optional[License] = None, + is_temp: bool = True, + content_type: Text = "text/csv", + content_encoding: Optional[Text] = None, + nattempts: int = 2, + return_download_link: bool = False) -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/file_utils.py#L97) + +Upload a file to S3 using pre-signed URLs with retry support. + +This function handles file uploads to S3 by first obtaining a pre-signed URL +from the aiXplain backend and then using it to upload the file. It supports +both temporary and permanent storage with optional metadata like tags and +license information. + +**Arguments**: + +- `file_name` _Union[Text, Path]_ - Local path of the file to upload. +- `tags` _Optional[List[Text]], optional_ - List of tags to associate with + the file. Only used when is_temp is False. Defaults to None. +- `license` _Optional[License], optional_ - License to associate with the file. + Only used when is_temp is False. Defaults to None. +- `is_temp` _bool, optional_ - Whether to upload as a temporary file. + Temporary files have different handling and URL generation. + Defaults to True. +- `content_type` _Text, optional_ - MIME type of the content being uploaded. + Defaults to "text/csv". +- `content_encoding` _Optional[Text], optional_ - Content encoding of the file + (e.g., 'gzip'). Defaults to None. +- `nattempts` _int, optional_ - Number of retry attempts for upload failures. + Defaults to 2. +- `return_download_link` _bool, optional_ - If True, returns a direct download + URL instead of the S3 path. Defaults to False. + + +**Returns**: + +- `str` - Either an S3 path (s3://bucket/key) or a download URL, depending + on return_download_link parameter. + + +**Raises**: + +- `Exception` - If the upload fails after all retry attempts. + + +**Notes**: + + The function will automatically retry failed uploads up to nattempts + times before raising an exception. + +#### s3\_to\_csv + +```python +def s3_to_csv( + s3_url: Text, + aws_credentials: Optional[Dict[Text, Text]] = { + "AWS_ACCESS_KEY_ID": None, + "AWS_SECRET_ACCESS_KEY": None + } +) -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/file_utils.py#L207) + +Convert S3 directory contents to a CSV file with file listings. + +This function takes an S3 URL and creates a CSV file containing listings +of all files in that location. It handles both single files and directories, +with special handling for directory structures. + +**Arguments**: + +- `s3_url` _Text_ - S3 URL in the format 's3://bucket-name/path'. +- `aws_credentials` _Optional[Dict[Text, Text]], optional_ - AWS credentials + dictionary with 'AWS_ACCESS_KEY_ID' and 'AWS_SECRET_ACCESS_KEY'. + If not provided or values are None, uses environment variables. + Defaults to \{"AWS_ACCESS_KEY_ID": None, "AWS_SECRET_ACCESS_KEY": None}. + + +**Returns**: + +- `str` - Path to the generated CSV file. The file contains listings of + all files found in the S3 location. + + +**Raises**: + +- `Exception` - If: + - boto3 is not installed + - Invalid S3 URL format + - AWS credentials are missing + - Bucket doesn't exist + - No files found + - Files are at bucket root + - Directory structure is invalid (unequal file counts or mismatched names) + + +**Notes**: + + - The function requires the boto3 package to be installed + - The generated CSV will have a UUID as filename + - For directory structures, all subdirectories must have the same + number of files with matching prefixes + diff --git a/docs/api-reference/python/aixplain/utils/init.md b/docs/api-reference/python/aixplain/utils/init.md new file mode 100644 index 00000000..b06bda9e --- /dev/null +++ b/docs/api-reference/python/aixplain/utils/init.md @@ -0,0 +1,25 @@ +--- +sidebar_label: utils +title: aixplain.utils +--- + +aiXplain SDK Library. +--- + +aiXplain SDK enables python programmers to add AI functions +to their software. + +Copyright 2022 The aiXplain SDK authors + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + diff --git a/docs/api-reference/python/aixplain/utils/llm_utils.md b/docs/api-reference/python/aixplain/utils/llm_utils.md new file mode 100644 index 00000000..90458345 --- /dev/null +++ b/docs/api-reference/python/aixplain/utils/llm_utils.md @@ -0,0 +1,30 @@ +--- +sidebar_label: llm_utils +title: aixplain.utils.llm_utils +--- + +#### get\_llm\_instance + +```python +def get_llm_instance(llm_id: Text, api_key: Optional[Text] = None) -> LLM +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/llm_utils.py#L6) + +Get an LLM instance with specific configuration. + +**Arguments**: + +- `llm_id` _Text_ - ID of the LLM model to use. +- `api_key` _Optional[Text], optional_ - API key to use. Defaults to None. + + +**Returns**: + +- `LLM` - Configured LLM instance. + + +**Raises**: + +- `Exception` - If the LLM model with the given ID is not found. + diff --git a/docs/api-reference/python/aixplain/utils/request_utils.md b/docs/api-reference/python/aixplain/utils/request_utils.md new file mode 100644 index 00000000..68b73173 --- /dev/null +++ b/docs/api-reference/python/aixplain/utils/request_utils.md @@ -0,0 +1,5 @@ +--- +sidebar_label: request_utils +title: aixplain.utils.request_utils +--- + diff --git a/docs/api-reference/python/aixplain/utils/validation_utils.md b/docs/api-reference/python/aixplain/utils/validation_utils.md new file mode 100644 index 00000000..9c434a43 --- /dev/null +++ b/docs/api-reference/python/aixplain/utils/validation_utils.md @@ -0,0 +1,74 @@ +--- +sidebar_label: validation_utils +title: aixplain.utils.validation_utils +--- + +#### dataset\_onboarding\_validation + +```python +def dataset_onboarding_validation(input_schema: List[Union[Dict, MetaData]], + output_schema: List[Union[Dict, MetaData]], + function: Function, + input_ref_data: Dict[Text, Any] = {}, + metadata_schema: List[Union[Dict, + MetaData]] = [], + content_path: Union[Union[Text, Path], + List[Union[Text, + Path]]] = [], + split_labels: Optional[List[Text]] = None, + split_rate: Optional[List[float]] = None, + s3_link: Optional[str] = None) -> None +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/utils/validation_utils.py#L27) + +Validate dataset parameters before onboarding. + +This function performs comprehensive validation of dataset parameters to ensure +they meet the requirements for onboarding. It checks: +- Input/output data type compatibility with the specified function +- Presence of required input data +- Validity of dataset splitting configuration +- Presence of content data source + +**Arguments**: + +- `input_schema` _List[Union[Dict, MetaData]]_ - Metadata describing the input + data structure and types. +- `output_schema` _List[Union[Dict, MetaData]]_ - Metadata describing the output + data structure and types. +- `function` _Function_ - The function type that this dataset is designed for + (e.g., translation, transcription). +- `input_ref_data` _Dict[Text, Any], optional_ - References to existing input + data in the platform. Defaults to \{}. +- `metadata_schema` _List[Union[Dict, MetaData]], optional_ - Additional metadata + describing the dataset. Defaults to []. + content_path (Union[Union[Text, Path], List[Union[Text, Path]]], optional): + Path(s) to local files containing the data. Defaults to []. +- `split_labels` _Optional[List[Text]], optional_ - Labels for dataset splits + (e.g., ["train", "test"]). Must be provided with split_rate. + Defaults to None. +- `split_rate` _Optional[List[float]], optional_ - Proportions for dataset splits + (e.g., [0.8, 0.2]). Must sum to 1.0 and match split_labels length. + Defaults to None. +- `s3_link` _Optional[str], optional_ - S3 URL to data files or directories. + Alternative to content_path. Defaults to None. + + +**Raises**: + +- `AssertionError` - If any validation fails: + - No input data specified + - Incompatible input/output types for function + - Invalid split configuration + - No content source provided + - Multiple split metadata entries + - Invalid split metadata type + - Mismatched split labels and rates + + +**Notes**: + + Either content_path or s3_link must be provided. If using splits, + both split_labels and split_rate must be provided. + diff --git a/docs/api-reference/python/aixplain/v2/agent.md b/docs/api-reference/python/aixplain/v2/agent.md new file mode 100644 index 00000000..ca646a2b --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/agent.md @@ -0,0 +1,114 @@ +--- +sidebar_label: agent +title: aixplain.v2.agent +--- + +### Agent Objects + +```python +class Agent(BaseResource, ListResourceMixin[BareListParams, "Agent"], + GetResourceMixin[BareGetParams, "Agent"]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/agent.py#L45) + +Resource for agents. + +**Attributes**: + +- `RESOURCE_PATH` - str: The resource path. +- `PAGINATE_PATH` - None: The path for pagination. +- `PAGINATE_METHOD` - str: The method for pagination. +- `PAGINATE_ITEMS_KEY` - None: The key for the response. + +#### create\_pipeline\_tool + +```python +@classmethod +def create_pipeline_tool(cls, + description: str, + pipeline: Union["Pipeline", str], + name: Optional[str] = None) -> "PipelineTool" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/agent.py#L107) + +Create a new pipeline tool. + +#### create\_python\_interpreter\_tool + +```python +@classmethod +def create_python_interpreter_tool(cls) -> "PythonInterpreterTool" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/agent.py#L116) + +Create a new python interpreter tool. + +#### create\_custom\_python\_code\_tool + +```python +@classmethod +def create_custom_python_code_tool( + cls, + code: Union[str, Callable], + name: str, + description: str = "") -> "CustomPythonCodeTool" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/agent.py#L123) + +Create a new custom python code tool. + +#### create\_sql\_tool + +```python +@classmethod +def create_sql_tool(cls, + name: str, + description: str, + source: str, + source_type: str, + schema: Optional[str] = None, + tables: Optional[List[str]] = None, + enable_commit: bool = False) -> "SQLTool" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/agent.py#L132) + +Create a new SQL tool. + +**Arguments**: + +- `description` _str_ - description of the database tool +- `source` _Union[str, Dict]_ - database source - can be a connection string or dictionary with connection details +- `source_type` _str_ - type of source (sqlite, csv) +- `schema` _Optional[str], optional_ - database schema description +- `tables` _Optional[List[str]], optional_ - table names to work with (optional) +- `enable_commit` _bool, optional_ - enable to modify the database (optional) + + +**Returns**: + +- `SQLTool` - created SQLTool + + +**Examples**: + + # SQLite - Simple + sql_tool = Agent.create_sql_tool( + description="My SQLite Tool", + source="/path/to/database.sqlite", + source_type="sqlite", + tables=["users", "products"] + ) + + # CSV - Simple + sql_tool = Agent.create_sql_tool( + description="My CSV Tool", + source="/path/to/data.csv", + source_type="csv", + tables=["data"] + ) + diff --git a/docs/api-reference/python/aixplain/v2/api_key.md b/docs/api-reference/python/aixplain/v2/api_key.md new file mode 100644 index 00000000..169f67cd --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/api_key.md @@ -0,0 +1,5 @@ +--- +sidebar_label: api_key +title: aixplain.v2.api_key +--- + diff --git a/docs/api-reference/python/aixplain/v2/benchmark.md b/docs/api-reference/python/aixplain/v2/benchmark.md new file mode 100644 index 00000000..1e260c81 --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/benchmark.md @@ -0,0 +1,80 @@ +--- +sidebar_label: benchmark +title: aixplain.v2.benchmark +--- + +### BenchmarkCreateParams Objects + +```python +class BenchmarkCreateParams(BareCreateParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/benchmark.py#L18) + +Parameters for creating a benchmark. + +**Attributes**: + +- `name` - str: The name of the benchmark. +- `dataset_list` - List["Dataset"]: The list of datasets. +- `model_list` - List["Model"]: The list of models. +- `metric_list` - List["Metric"]: The list of metrics. + +### Benchmark Objects + +```python +class Benchmark(BaseResource, GetResourceMixin[BareGetParams, "Benchmark"], + CreateResourceMixin[BenchmarkCreateParams, "Benchmark"]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/benchmark.py#L34) + +Resource for benchmarks. + +#### list\_normalization\_options + +```python +@classmethod +def list_normalization_options(cls, metric: "Metric", + model: "Model") -> List[str] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/benchmark.py#L56) + +List the normalization options for a metric and a model. + +**Arguments**: + +- `metric` - "Metric": The metric. +- `model` - "Model": The model. + + +**Returns**: + +- `List[str]` - The list of normalization options. + +### BenchmarkJob Objects + +```python +class BenchmarkJob(BaseResource, GetResourceMixin[BareGetParams, + "BenchmarkJob"]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/benchmark.py#L72) + +Resource for benchmark jobs. + +#### get\_scores + +```python +def get_scores() -> dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/benchmark.py#L86) + +Get the scores for a benchmark job. + +**Returns**: + +- `dict` - The scores. + diff --git a/docs/api-reference/python/aixplain/v2/client.md b/docs/api-reference/python/aixplain/v2/client.md new file mode 100644 index 00000000..b08da05c --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/client.md @@ -0,0 +1,113 @@ +--- +sidebar_label: client +title: aixplain.v2.client +--- + +#### create\_retry\_session + +```python +def create_retry_session(total=None, + backoff_factor=None, + status_forcelist=None, + **kwargs) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/client.py#L13) + +Creates a requests.Session with a specified retry strategy. + +**Arguments**: + +- `total` _int, optional_ - Total number of retries allowed. Defaults to 5. +- `backoff_factor` _float, optional_ - Backoff factor to apply between retry attempts. Defaults to 0.1. +- `status_forcelist` _list, optional_ - List of HTTP status codes to force a retry on. Defaults to [500, 502, 503, 504]. +- `kwargs` _dict, optional_ - Additional keyword arguments for internal Retry object. + + +**Returns**: + +- `requests.Session` - A requests.Session object with the specified retry strategy. + +### AixplainClient Objects + +```python +class AixplainClient() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/client.py#L43) + +#### \_\_init\_\_ + +```python +def __init__(base_url: str, + aixplain_api_key: str = None, + team_api_key: str = None, + retry_total=DEFAULT_RETRY_TOTAL, + retry_backoff_factor=DEFAULT_RETRY_BACKOFF_FACTOR, + retry_status_forcelist=DEFAULT_RETRY_STATUS_FORCELIST) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/client.py#L44) + +Initializes AixplainClient with authentication and retry configuration. + +**Arguments**: + +- `base_url` _str_ - The base URL for the API. +- `aixplain_api_key` _str, optional_ - The individual API key. +- `team_api_key` _str, optional_ - The team API key. +- `retry_total` _int, optional_ - Total number of retries allowed. Defaults to None, uses DEFAULT_RETRY_TOTAL. +- `retry_backoff_factor` _float, optional_ - Backoff factor to apply between retry attempts. Defaults to None, uses DEFAULT_RETRY_BACKOFF_FACTOR. +- `retry_status_forcelist` _list, optional_ - List of HTTP status codes to force a retry on. Defaults to None, uses DEFAULT_RETRY_STATUS_FORCELIST. + +#### request + +```python +def request(method: str, path: str, **kwargs: Any) -> requests.Response +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/client.py#L87) + +Sends an HTTP request. + +**Arguments**: + +- `method` _str_ - HTTP method (e.g. 'GET', 'POST') +- `path` _str_ - URL path +- `kwargs` _dict, optional_ - Additional keyword arguments for the request + + +**Returns**: + +- `requests.Response` - The response from the request + +#### get + +```python +def get(path: str, **kwargs: Any) -> requests.Response +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/client.py#L104) + +Sends an HTTP GET request. + +**Arguments**: + +- `path` _str_ - URL path +- `kwargs` _dict, optional_ - Additional keyword arguments for the request + + +**Returns**: + +- `requests.Response` - The response from the request + +#### get\_obj + +```python +def get_obj(path: str, **kwargs: Any) -> dict +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/client.py#L117) + +Sends an HTTP GET request and returns the object. + diff --git a/docs/api-reference/python/aixplain/v2/core.md b/docs/api-reference/python/aixplain/v2/core.md new file mode 100644 index 00000000..a31d0639 --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/core.md @@ -0,0 +1,97 @@ +--- +sidebar_label: core +title: aixplain.v2.core +--- + +### Aixplain Objects + +```python +class Aixplain() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/core.py#L37) + +Main class for the Aixplain API. + +**Attributes**: + +- `_instance` - Aixplain: The unique instance of the Aixplain class. +- `api_key` - str: The API key for the Aixplain API. +- `base_url` - str: The URL for the backend. +- `pipeline_url` - str: The URL for the pipeline. +- `model_url` - str: The URL for the model. +- `client` - AixplainClient: The client for the Aixplain API. +- `Model` - type: The model class. +- `Pipeline` - type: The pipeline class. +- `Agent` - type: The agent class. +- `Benchmark` - type: The benchmark class. +- `api_key`0 - type: The benchmark job class. + +#### \_\_new\_\_ + +```python +def __new__(cls, *args, **kwargs) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/core.py#L100) + +Singleton pattern for the Aixplain class. +Otherwise, the environment variables will be overwritten in multiple instances. + +TODO: This should be removed once the factory classes are removed. + +#### \_\_init\_\_ + +```python +def __init__(api_key: str = None, + backend_url: str = None, + pipeline_url: str = None, + model_url: str = None) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/core.py#L112) + +Initialize the Aixplain class. + +**Arguments**: + +- `api_key` - str: The API key for the Aixplain API. +- `backend_url` - str: The URL for the backend. +- `pipeline_url` - str: The URL for the pipeline. +- `model_url` - str: The URL for the model. + +#### init\_client + +```python +def init_client() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/core.py#L140) + +Initialize the client. + +#### init\_env + +```python +def init_env() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/core.py#L147) + +Initialize the environment variables. + +This is required for the legacy use of the factory classes. + +#### init\_resources + +```python +def init_resources() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/core.py#L157) + +Initialize the resources. + +We're dynamically creating the classes here to avoid potential race +conditions when using class level attributes + diff --git a/docs/api-reference/python/aixplain/v2/corpus.md b/docs/api-reference/python/aixplain/v2/corpus.md new file mode 100644 index 00000000..1ae29950 --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/corpus.md @@ -0,0 +1,24 @@ +--- +sidebar_label: corpus +title: aixplain.v2.corpus +--- + +### CorpusListParams Objects + +```python +class CorpusListParams(BaseListParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/corpus.py#L34) + +Parameters for listing corpora. + +**Attributes**: + +- `query` - Optional[Text]: A search query. +- `function` - Optional[Function]: The function of the model. +- `suppliers` - Union[Supplier, List[Supplier]: The suppliers of the model. +- `source_languages` - Union[Language, List[Language]: The source languages of the model. +- `target_languages` - Union[Language, List[Language]: The target languages of the model. +- `is_finetunable` - bool: Whether the model is finetunable. + diff --git a/docs/api-reference/python/aixplain/v2/data.md b/docs/api-reference/python/aixplain/v2/data.md new file mode 100644 index 00000000..2c304907 --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/data.md @@ -0,0 +1,5 @@ +--- +sidebar_label: data +title: aixplain.v2.data +--- + diff --git a/docs/api-reference/python/aixplain/v2/dataset.md b/docs/api-reference/python/aixplain/v2/dataset.md new file mode 100644 index 00000000..91255a42 --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/dataset.md @@ -0,0 +1,24 @@ +--- +sidebar_label: dataset +title: aixplain.v2.dataset +--- + +### DatasetListParams Objects + +```python +class DatasetListParams(BaseListParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/dataset.py#L48) + +Parameters for listing corpora. + +**Attributes**: + +- `query` - Optional[Text]: A search query. +- `function` - Optional[Function]: The function of the model. +- `suppliers` - Union[Supplier, List[Supplier]: The suppliers of the model. +- `source_languages` - Union[Language, List[Language]: The source languages of the model. +- `target_languages` - Union[Language, List[Language]: The target languages of the model. +- `is_finetunable` - bool: Whether the model is finetunable. + diff --git a/docs/api-reference/python/aixplain/v2/enums.md b/docs/api-reference/python/aixplain/v2/enums.md new file mode 100644 index 00000000..abb096a1 --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/enums.md @@ -0,0 +1,5 @@ +--- +sidebar_label: enums +title: aixplain.v2.enums +--- + diff --git a/docs/api-reference/python/aixplain/v2/enums_include.md b/docs/api-reference/python/aixplain/v2/enums_include.md new file mode 100644 index 00000000..0f15bb4f --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/enums_include.md @@ -0,0 +1,20 @@ +--- +sidebar_label: enums_include +title: aixplain.v2.enums_include +--- + +### ErrorHandler Objects + +```python +class ErrorHandler(str, Enum) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/enums_include.py#L54) + +Enumeration class defining different error handler strategies. + +**Attributes**: + +- `SKIP` _str_ - skip failed rows. +- `FAIL` _str_ - raise an exception. + diff --git a/docs/api-reference/python/aixplain/v2/file.md b/docs/api-reference/python/aixplain/v2/file.md new file mode 100644 index 00000000..8c032b3c --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/file.md @@ -0,0 +1,100 @@ +--- +sidebar_label: file +title: aixplain.v2.file +--- + +### FileCreateParams Objects + +```python +class FileCreateParams(BaseCreateParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/file.py#L10) + +Parameters for creating a file. + +### File Objects + +```python +class File(BaseResource, CreateResourceMixin[FileCreateParams, "File"]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/file.py#L19) + +Resource for files. + +#### create + +```python +@classmethod +def create(cls, *args, **kwargs: Unpack[FileCreateParams]) -> "File" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/file.py#L25) + +Create a file. + +#### to\_link + +```python +@classmethod +def to_link(cls, local_path: str) -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/file.py#L36) + +Convert a local path to a link. + +**Arguments**: + +- `local_path` - str: The local path to the file. + + +**Returns**: + +- `str` - The link to the file. + +#### upload + +```python +@classmethod +def upload(cls, + local_path: str, + tags: List[str] = None, + license: "License" = None, + is_temp: bool = True) -> str +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/file.py#L50) + +Upload a file. + +**Arguments**: + +- `local_path` - str: The local path to the file. + + +**Returns**: + +- `str` - The upload URL. + +#### check\_storage\_type + +```python +@classmethod +def check_storage_type(cls, upload_url: str) -> "StorageType" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/file.py#L70) + +Check the storage type of a file. + +**Arguments**: + +- `upload_url` - str: The upload URL. + + +**Returns**: + +- `StorageType` - The storage type of the file. + diff --git a/docs/api-reference/python/aixplain/v2/finetune.md b/docs/api-reference/python/aixplain/v2/finetune.md new file mode 100644 index 00000000..d4e84591 --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/finetune.md @@ -0,0 +1,36 @@ +--- +sidebar_label: finetune +title: aixplain.v2.finetune +--- + +### FinetuneCreateParams Objects + +```python +class FinetuneCreateParams(BareCreateParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/finetune.py#L16) + +Parameters for creating a finetune. + +**Attributes**: + +- `name` - str: The name of the finetune. +- `dataset_list` - List[Dataset]: The list of datasets. +- `model` - Union[Model, str]: The model. +- `prompt_template` - str: The prompt template. +- `hyperparameters` - Hyperparameters: The hyperparameters. +- `train_percentage` - float: The train percentage. +- `dev_percentage` - float: The dev percentage. + +### Finetune Objects + +```python +class Finetune(BaseResource, CreateResourceMixin[FinetuneCreateParams, + "Finetune"]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/finetune.py#L38) + +Resource for finetunes. + diff --git a/docs/api-reference/python/aixplain/v2/init.md b/docs/api-reference/python/aixplain/v2/init.md new file mode 100644 index 00000000..26dcfb1c --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/init.md @@ -0,0 +1,6 @@ +--- +draft: true +sidebar_label: v2 +title: aixplain.v2 +--- + diff --git a/docs/api-reference/python/aixplain/v2/metric.md b/docs/api-reference/python/aixplain/v2/metric.md new file mode 100644 index 00000000..12d88089 --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/metric.md @@ -0,0 +1,32 @@ +--- +sidebar_label: metric +title: aixplain.v2.metric +--- + +### MetricListParams Objects + +```python +class MetricListParams(BaseListParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/metric.py#L13) + +Parameters for listing metrics. + +**Attributes**: + +- `model_id` - str: The model ID. +- `is_source_required` - bool: Whether the source is required. +- `is_reference_required` - bool: Whether the reference is required. + +### Metric Objects + +```python +class Metric(BaseResource, ListResourceMixin[MetricListParams, "Metric"], + GetResourceMixin[BareGetParams, "Metric"]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/metric.py#L27) + +Resource for metrics. + diff --git a/docs/api-reference/python/aixplain/v2/model.md b/docs/api-reference/python/aixplain/v2/model.md new file mode 100644 index 00000000..75cf191f --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/model.md @@ -0,0 +1,34 @@ +--- +sidebar_label: model +title: aixplain.v2.model +--- + +### ModelListParams Objects + +```python +class ModelListParams(BaseListParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/model.py#L18) + +Parameters for listing models. + +**Attributes**: + +- `function` - Function: The function of the model. +- `suppliers` - Union[Supplier, List[Supplier]: The suppliers of the model. +- `source_languages` - Union[Language, List[Language]: The source languages of the model. +- `target_languages` - Union[Language, List[Language]: The target languages of the model. +- `is_finetunable` - bool: Whether the model is finetunable. + +### Model Objects + +```python +class Model(BaseResource, ListResourceMixin[ModelListParams, "Model"], + GetResourceMixin[BareGetParams, "Model"]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/model.py#L36) + +Resource for models. + diff --git a/docs/api-reference/python/aixplain/v2/pipeline.md b/docs/api-reference/python/aixplain/v2/pipeline.md new file mode 100644 index 00000000..bf73a69c --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/pipeline.md @@ -0,0 +1,40 @@ +--- +sidebar_label: pipeline +title: aixplain.v2.pipeline +--- + +### PipelineListParams Objects + +```python +class PipelineListParams(BareListParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/pipeline.py#L18) + +Parameters for listing pipelines. + +**Attributes**: + +- `functions` - Union[Function, List[Function]]: The functions of the pipeline. +- `suppliers` - Union[Supplier, List[Supplier]]: The suppliers of the pipeline. +- `models` - Union[Model, List[Model]]: The models of the pipeline. +- `input_data_types` - Union[DataType, List[DataType]]: The input data types of the pipeline. +- `output_data_types` - Union[DataType, List[DataType]]: The output data types of the pipeline. +- `drafts_only` - bool: Whether to list only drafts. + +### Pipeline Objects + +```python +class Pipeline(BaseResource, ListResourceMixin[PipelineListParams, "Pipeline"], + GetResourceMixin[BareGetParams, "Pipeline"], + CreateResourceMixin[PipelineCreateParams, "Pipeline"]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/pipeline.py#L44) + +Resource for pipelines. + +**Attributes**: + +- `RESOURCE_PATH` - str: The resource path. + diff --git a/docs/api-reference/python/aixplain/v2/resource.md b/docs/api-reference/python/aixplain/v2/resource.md new file mode 100644 index 00000000..d160c5ff --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/resource.md @@ -0,0 +1,270 @@ +--- +sidebar_label: resource +title: aixplain.v2.resource +--- + +### BaseResource Objects + +```python +class BaseResource() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L23) + +Base class for all resources. + +**Attributes**: + +- `context` - Aixplain: The Aixplain instance. +- `RESOURCE_PATH` - str: The resource path. + +#### \_\_init\_\_ + +```python +def __init__(obj: Union[dict, Any]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L35) + +Initialize a BaseResource instance. + +**Arguments**: + +- `obj` - dict: Dictionary containing the resource's attributes. + +#### \_\_getattr\_\_ + +```python +def __getattr__(key: str) -> Any +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L44) + +Return the value corresponding to the key from the wrapped dictionary +if found, otherwise raise an AttributeError. + +**Arguments**: + +- `key` - str: Attribute name to retrieve from the resource. + + +**Returns**: + +- `Any` - Value corresponding to the specified key. + + +**Raises**: + +- `AttributeError` - If the key is not found in the wrapped + dictionary. + +#### save + +```python +def save() +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L65) + +Save the resource. + +If the resource has an ID, it will be updated, otherwise it will be created. + +### BaseListParams Objects + +```python +class BaseListParams(TypedDict) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L111) + +Base class for all list parameters. + +**Attributes**: + +- `query` - str: The query string. +- `ownership` - Tuple[OwnershipType, List[OwnershipType]]: The ownership type. +- `sort_by` - SortBy: The attribute to sort by. +- `sort_order` - SortOrder: The order to sort by. +- `page_number` - int: The page number. +- `page_size` - int: The page size. + +### BaseGetParams Objects + +```python +class BaseGetParams(TypedDict) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L131) + +Base class for all get parameters. + +**Attributes**: + +- `id` - str: The resource ID. + +### BaseCreateParams Objects + +```python +class BaseCreateParams(TypedDict) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L141) + +Base class for all create parameters. + +**Attributes**: + +- `name` - str: The name of the resource. + +### BareCreateParams Objects + +```python +class BareCreateParams(BaseCreateParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L151) + +Default implementation of create parameters. + +### BareListParams Objects + +```python +class BareListParams(BaseListParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L157) + +Default implementation of list parameters. + +### BareGetParams Objects + +```python +class BareGetParams(BaseGetParams) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L163) + +Default implementation of get parameters. + +### Page Objects + +```python +class Page(Generic[R]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L175) + +Page of resources. + +**Attributes**: + +- `items` - List[R]: The list of resources. +- `total` - int: The total number of resources. + +### ListResourceMixin Objects + +```python +class ListResourceMixin(Generic[L, R]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L201) + +Mixin for listing resources. + +**Attributes**: + +- `PAGINATE_PATH` - str: The path for pagination. +- `PAGINATE_METHOD` - str: The method for pagination. +- `PAGINATE_ITEMS_KEY` - str: The key for the response. +- `PAGINATE_TOTAL_KEY` - str: The key for the total number of resources. +- `PAGINATE_PAGE_TOTAL_KEY` - str: The key for the total number of pages. +- `PAGINATE_DEFAULT_PAGE_NUMBER` - int: The default page number. +- `PAGINATE_DEFAULT_PAGE_SIZE` - int: The default page size. + +#### list + +```python +@classmethod +def list(cls: Type[R], **kwargs: Unpack[L]) -> Page[R] +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L224) + +List resources across the first n pages with optional filtering. + +**Arguments**: + +- `kwargs` - Unpack[L]: The keyword arguments. + + +**Returns**: + +- `Page[R]` - Page of BaseResource instances + +### GetResourceMixin Objects + +```python +class GetResourceMixin(Generic[G, R]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L334) + +Mixin for getting a resource. + +#### get + +```python +@classmethod +def get(cls: Type[R], id: Any, **kwargs: Unpack[G]) -> R +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L338) + +Retrieve a single resource by its ID (or other get parameters). + +**Arguments**: + +- `id` - Any: The ID of the resource to get. +- `kwargs` - Unpack[G]: Get parameters to pass to the request. + + +**Returns**: + +- `BaseResource` - Instance of the BaseResource class. + + +**Raises**: + +- `ValueError` - If 'RESOURCE_PATH' is not defined by the subclass. + +### CreateResourceMixin Objects + +```python +class CreateResourceMixin(Generic[C, R]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L359) + +Mixin for creating a resource. + +#### create + +```python +@classmethod +def create(cls, *args, **kwargs: Unpack[C]) -> R +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/resource.py#L363) + +Create a resource. + +**Arguments**: + +- `kwargs` - Unpack[C]: The keyword arguments. + + +**Returns**: + +- `BaseResource` - The created resource. + diff --git a/docs/api-reference/python/aixplain/v2/script.md b/docs/api-reference/python/aixplain/v2/script.md new file mode 100644 index 00000000..4558edcf --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/script.md @@ -0,0 +1,33 @@ +--- +sidebar_label: script +title: aixplain.v2.script +--- + +### Script Objects + +```python +class Script(BaseResource) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/script.py#L4) + +#### upload + +```python +@classmethod +def upload(cls, script_path: str) -> "Script" +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/script.py#L6) + +Upload a script to the server. + +**Arguments**: + +- `script_path` - str: The path to the script. + + +**Returns**: + +- `Script` - The script. + diff --git a/docs/api-reference/python/aixplain/v2/team_agent.md b/docs/api-reference/python/aixplain/v2/team_agent.md new file mode 100644 index 00000000..64ddabe5 --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/team_agent.md @@ -0,0 +1,23 @@ +--- +sidebar_label: team_agent +title: aixplain.v2.team_agent +--- + +### TeamAgent Objects + +```python +class TeamAgent(BaseResource, ListResourceMixin[BareListParams, "TeamAgent"], + GetResourceMixin[BareGetParams, "TeamAgent"]) +``` + +[[view_source]](https://github.com/aixplain/aiXplain/blob/main/aixplain/v2/team_agent.py#L41) + +Resource for agents. + +**Attributes**: + +- `RESOURCE_PATH` - str: The resource path. +- `PAGINATE_PATH` - None: The path for pagination. +- `PAGINATE_METHOD` - str: The method for pagination. +- `PAGINATE_ITEMS_KEY` - None: The key for the response. + diff --git a/docs/api-reference/python/aixplain/v2/wallet.md b/docs/api-reference/python/aixplain/v2/wallet.md new file mode 100644 index 00000000..29b9f823 --- /dev/null +++ b/docs/api-reference/python/aixplain/v2/wallet.md @@ -0,0 +1,5 @@ +--- +sidebar_label: wallet +title: aixplain.v2.wallet +--- + diff --git a/docs/api-reference/python/api_sidebar.js b/docs/api-reference/python/api_sidebar.js new file mode 100644 index 00000000..476b437f --- /dev/null +++ b/docs/api-reference/python/api_sidebar.js @@ -0,0 +1,321 @@ +{ + "items": [ + { + "items": [ + { + "items": [ + "api-reference/python/aixplain/base/parameters" + ], + "label": "aixplain.base", + "type": "category" + }, + { + "items": [ + "api-reference/python/aixplain/decorators/init", + "api-reference/python/aixplain/decorators/api_key_checker" + ], + "label": "aixplain.decorators", + "type": "category" + }, + { + "items": [ + "api-reference/python/aixplain/enums/init", + "api-reference/python/aixplain/enums/asset_status", + "api-reference/python/aixplain/enums/code_interpreter", + "api-reference/python/aixplain/enums/data_split", + "api-reference/python/aixplain/enums/data_subtype", + "api-reference/python/aixplain/enums/data_type", + "api-reference/python/aixplain/enums/database_source", + "api-reference/python/aixplain/enums/embedding_model", + "api-reference/python/aixplain/enums/error_handler", + "api-reference/python/aixplain/enums/file_type", + "api-reference/python/aixplain/enums/function", + "api-reference/python/aixplain/enums/function_type", + "api-reference/python/aixplain/enums/index_stores", + "api-reference/python/aixplain/enums/language", + "api-reference/python/aixplain/enums/license", + "api-reference/python/aixplain/enums/onboard_status", + "api-reference/python/aixplain/enums/ownership_type", + "api-reference/python/aixplain/enums/privacy", + "api-reference/python/aixplain/enums/response_status", + "api-reference/python/aixplain/enums/sort_by", + "api-reference/python/aixplain/enums/sort_order", + "api-reference/python/aixplain/enums/splitting_options", + "api-reference/python/aixplain/enums/status", + "api-reference/python/aixplain/enums/storage_type", + "api-reference/python/aixplain/enums/supplier" + ], + "label": "aixplain.enums", + "type": "category" + }, + { + "items": [ + "api-reference/python/aixplain/exceptions/init", + "api-reference/python/aixplain/exceptions/types" + ], + "label": "aixplain.exceptions", + "type": "category" + }, + { + "items": [ + { + "items": [ + "api-reference/python/aixplain/factories/agent_factory/init", + "api-reference/python/aixplain/factories/agent_factory/utils" + ], + "label": "aixplain.factories.agent_factory", + "type": "category" + }, + { + "items": [ + "api-reference/python/aixplain/factories/cli/model_factory_cli" + ], + "label": "aixplain.factories.cli", + "type": "category" + }, + { + "items": [ + "api-reference/python/aixplain/factories/finetune_factory/init", + "api-reference/python/aixplain/factories/finetune_factory/prompt_validator" + ], + "label": "aixplain.factories.finetune_factory", + "type": "category" + }, + { + "items": [ + "api-reference/python/aixplain/factories/index_factory/init", + "api-reference/python/aixplain/factories/index_factory/utils" + ], + "label": "aixplain.factories.index_factory", + "type": "category" + }, + { + "items": [ + { + "items": [ + "api-reference/python/aixplain/factories/model_factory/mixins/init", + "api-reference/python/aixplain/factories/model_factory/mixins/model_getter", + "api-reference/python/aixplain/factories/model_factory/mixins/model_list" + ], + "label": "aixplain.factories.model_factory.mixins", + "type": "category" + }, + "api-reference/python/aixplain/factories/model_factory/init", + "api-reference/python/aixplain/factories/model_factory/utils" + ], + "label": "aixplain.factories.model_factory", + "type": "category" + }, + { + "items": [ + "api-reference/python/aixplain/factories/pipeline_factory/init", + "api-reference/python/aixplain/factories/pipeline_factory/utils" + ], + "label": "aixplain.factories.pipeline_factory", + "type": "category" + }, + { + "items": [ + "api-reference/python/aixplain/factories/team_agent_factory/init", + "api-reference/python/aixplain/factories/team_agent_factory/inspector_factory", + "api-reference/python/aixplain/factories/team_agent_factory/utils" + ], + "label": "aixplain.factories.team_agent_factory", + "type": "category" + }, + "api-reference/python/aixplain/factories/init", + "api-reference/python/aixplain/factories/api_key_factory", + "api-reference/python/aixplain/factories/asset_factory", + "api-reference/python/aixplain/factories/benchmark_factory", + "api-reference/python/aixplain/factories/corpus_factory", + "api-reference/python/aixplain/factories/data_factory", + "api-reference/python/aixplain/factories/dataset_factory", + "api-reference/python/aixplain/factories/file_factory", + "api-reference/python/aixplain/factories/integration_factory", + "api-reference/python/aixplain/factories/metric_factory", + "api-reference/python/aixplain/factories/script_factory", + "api-reference/python/aixplain/factories/tool_factory", + "api-reference/python/aixplain/factories/wallet_factory" + ], + "label": "aixplain.factories", + "type": "category" + }, + { + "items": [ + { + "items": [ + { + "items": [ + "api-reference/python/aixplain/modules/agent/tool/init", + "api-reference/python/aixplain/modules/agent/tool/custom_python_code_tool", + "api-reference/python/aixplain/modules/agent/tool/model_tool", + "api-reference/python/aixplain/modules/agent/tool/pipeline_tool", + "api-reference/python/aixplain/modules/agent/tool/python_interpreter_tool", + "api-reference/python/aixplain/modules/agent/tool/sql_tool" + ], + "label": "aixplain.modules.agent.tool", + "type": "category" + }, + "api-reference/python/aixplain/modules/agent/init", + "api-reference/python/aixplain/modules/agent/agent_response", + "api-reference/python/aixplain/modules/agent/agent_response_data", + "api-reference/python/aixplain/modules/agent/agent_task", + "api-reference/python/aixplain/modules/agent/model_with_params", + "api-reference/python/aixplain/modules/agent/output_format", + "api-reference/python/aixplain/modules/agent/utils" + ], + "label": "aixplain.modules.agent", + "type": "category" + }, + { + "items": [ + "api-reference/python/aixplain/modules/finetune/init", + "api-reference/python/aixplain/modules/finetune/cost", + "api-reference/python/aixplain/modules/finetune/hyperparameters", + "api-reference/python/aixplain/modules/finetune/status" + ], + "label": "aixplain.modules.finetune", + "type": "category" + }, + { + "items": [ + "api-reference/python/aixplain/modules/model/init", + "api-reference/python/aixplain/modules/model/connection", + "api-reference/python/aixplain/modules/model/index_model", + "api-reference/python/aixplain/modules/model/integration", + "api-reference/python/aixplain/modules/model/llm_model", + "api-reference/python/aixplain/modules/model/mcp_connection", + "api-reference/python/aixplain/modules/model/model_parameters", + "api-reference/python/aixplain/modules/model/model_response_streamer", + "api-reference/python/aixplain/modules/model/record", + "api-reference/python/aixplain/modules/model/response", + "api-reference/python/aixplain/modules/model/utility_model", + "api-reference/python/aixplain/modules/model/utils" + ], + "label": "aixplain.modules.model", + "type": "category" + }, + { + "items": [ + { + "items": [ + "api-reference/python/aixplain/modules/pipeline/designer/init", + "api-reference/python/aixplain/modules/pipeline/designer/base", + "api-reference/python/aixplain/modules/pipeline/designer/enums", + "api-reference/python/aixplain/modules/pipeline/designer/mixins", + "api-reference/python/aixplain/modules/pipeline/designer/nodes", + "api-reference/python/aixplain/modules/pipeline/designer/pipeline", + "api-reference/python/aixplain/modules/pipeline/designer/utils" + ], + "label": "aixplain.modules.pipeline.designer", + "type": "category" + }, + "api-reference/python/aixplain/modules/pipeline/init", + "api-reference/python/aixplain/modules/pipeline/asset", + "api-reference/python/aixplain/modules/pipeline/default", + "api-reference/python/aixplain/modules/pipeline/pipeline", + "api-reference/python/aixplain/modules/pipeline/response" + ], + "label": "aixplain.modules.pipeline", + "type": "category" + }, + { + "items": [ + "api-reference/python/aixplain/modules/team_agent/init", + "api-reference/python/aixplain/modules/team_agent/inspector" + ], + "label": "aixplain.modules.team_agent", + "type": "category" + }, + "api-reference/python/aixplain/modules/init", + "api-reference/python/aixplain/modules/api_key", + "api-reference/python/aixplain/modules/asset", + "api-reference/python/aixplain/modules/benchmark", + "api-reference/python/aixplain/modules/benchmark_job", + "api-reference/python/aixplain/modules/content_interval", + "api-reference/python/aixplain/modules/corpus", + "api-reference/python/aixplain/modules/data", + "api-reference/python/aixplain/modules/dataset", + "api-reference/python/aixplain/modules/file", + "api-reference/python/aixplain/modules/metadata", + "api-reference/python/aixplain/modules/metric", + "api-reference/python/aixplain/modules/mixins", + "api-reference/python/aixplain/modules/wallet" + ], + "label": "aixplain.modules", + "type": "category" + }, + { + "items": [ + { + "items": [ + "api-reference/python/aixplain/processes/data_onboarding/init", + "api-reference/python/aixplain/processes/data_onboarding/onboard_functions", + "api-reference/python/aixplain/processes/data_onboarding/process_media_files", + "api-reference/python/aixplain/processes/data_onboarding/process_text_files" + ], + "label": "aixplain.processes.data_onboarding", + "type": "category" + }, + "api-reference/python/aixplain/processes/init" + ], + "label": "aixplain.processes", + "type": "category" + }, + { + "items": [ + "api-reference/python/aixplain/utils/init", + "api-reference/python/aixplain/utils/asset_cache", + "api-reference/python/aixplain/utils/cache_utils", + "api-reference/python/aixplain/utils/config", + "api-reference/python/aixplain/utils/convert_datatype_utils", + "api-reference/python/aixplain/utils/file_utils", + "api-reference/python/aixplain/utils/llm_utils", + "api-reference/python/aixplain/utils/request_utils", + "api-reference/python/aixplain/utils/validation_utils" + ], + "label": "aixplain.utils", + "type": "category" + }, + { + "items": [ + "api-reference/python/aixplain/v2/init", + "api-reference/python/aixplain/v2/agent", + "api-reference/python/aixplain/v2/api_key", + "api-reference/python/aixplain/v2/benchmark", + "api-reference/python/aixplain/v2/client", + "api-reference/python/aixplain/v2/core", + "api-reference/python/aixplain/v2/corpus", + "api-reference/python/aixplain/v2/data", + "api-reference/python/aixplain/v2/dataset", + "api-reference/python/aixplain/v2/enums", + "api-reference/python/aixplain/v2/enums_include", + "api-reference/python/aixplain/v2/file", + "api-reference/python/aixplain/v2/finetune", + "api-reference/python/aixplain/v2/metric", + "api-reference/python/aixplain/v2/model", + "api-reference/python/aixplain/v2/pipeline", + "api-reference/python/aixplain/v2/resource", + "api-reference/python/aixplain/v2/script", + "api-reference/python/aixplain/v2/team_agent", + "api-reference/python/aixplain/v2/wallet" + ], + "label": "aixplain.v2", + "type": "category" + }, + "api-reference/python/aixplain/init", + "api-reference/python/aixplain/cli_groups" + ], + "label": "aixplain", + "type": "category", + "collapsible": false + } + ], + "label": "Python API reference", + "type": "category", + "collapsible": false, + "link": { + "type": "doc", + "id": "api-reference/python/python" + } +} \ No newline at end of file diff --git a/docs/assets/aixplain-workflow-teamagent.png b/docs/assets/aixplain-workflow-teamagent.png new file mode 100644 index 00000000..2259328d Binary files /dev/null and b/docs/assets/aixplain-workflow-teamagent.png differ diff --git a/post_process_docs.py b/post_process_docs.py new file mode 100644 index 00000000..7e64e09e --- /dev/null +++ b/post_process_docs.py @@ -0,0 +1,173 @@ +#!/usr/bin/env python3 + +import os +import re +import json + +def rename_files(docs_dir='docs/api-reference/python'): + """ + 1. Rename __init__.md files to init.md + 2. Remove leading underscores from filenames + """ + renamed_init_files = 0 + renamed_underscore_files = 0 + + # Walk through the docs directory + for root, _, files in os.walk(docs_dir): + for file in files: + # Rename __init__.md to init.md + if file == "__init__.md": + old_path = os.path.join(root, file) + new_path = os.path.join(root, "init.md") + os.rename(old_path, new_path) + renamed_init_files += 1 + + # Remove leading underscore from filenames + elif file.startswith('_') and file != "__init__.md": + old_path = os.path.join(root, file) + new_path = os.path.join(root, file[1:]) + os.rename(old_path, new_path) + renamed_underscore_files += 1 + + print(f"Renamed {renamed_init_files} __init__.md files to init.md") + print(f"Renamed {renamed_underscore_files} files by removing leading underscore") + +def process_content(docs_dir='docs/api-reference/python'): + """ + Process markdown content: + 1. Escape braces outside code blocks + """ + modified_files = 0 + + # Walk through the docs directory + for root, _, files in os.walk(docs_dir): + for file in files: + if file.endswith('.md'): + file_path = os.path.join(root, file) + + # Read the file + with open(file_path, 'r') as f: + content = f.read() + + # Process content + original_content = content + + # Escape braces outside code blocks + parts = re.split(r'(```.*?```)', content, flags=re.DOTALL) + for i in range(len(parts)): + if i % 2 == 0: # Outside code blocks + parts[i] = re.sub(r'(? init) + 2. Make top-level categories non-collapsible + 3. Add landing page link + """ + # Read the sidebar file + with open(sidebar_path, 'r') as f: + content = f.read() + + # 1. Fix sidebar references + init_replacements = content.count('__init__') + content = re.sub(r'/__init__"', r'/init"', content) + + # Write the intermediate changes + with open(sidebar_path, 'w') as f: + f.write(content) + + # Read as JSON for structural changes + with open(sidebar_path, 'r') as f: + sidebar_data = json.load(f) + + # 2. Make top-level categories non-collapsible + sidebar_data["collapsible"] = False + for item in sidebar_data.get("items", []): + if isinstance(item, dict) and item.get("type") == "category": + item["collapsible"] = False + + # 3. Add landing page link + sidebar_data["link"] = { + "type": "doc", + "id": "api-reference/python/python" + } + + # Write back the modified JSON + with open(sidebar_path, 'w') as f: + json.dump(sidebar_data, f, indent=2) + + print(f"Updated {init_replacements} __init__ references to init in sidebar") + print(f"Added collapsible: false to top-level categories") + print(f"Added landing page link to sidebar") + +def main(): + """ + Execute all post-processing steps for documentation + """ + print("Starting documentation post-processing...") + + # Create docs directory if it doesn't exist + os.makedirs('docs/api-reference/python', exist_ok=True) + + # 1. Rename files + rename_files() + + # 2. Process content + process_content() + + # 3. Mark empty init files + mark_empty_init_files() + + # 4. Configure sidebar + configure_sidebar() + + print("Documentation post-processing complete!") + +if __name__ == "__main__": + main() diff --git a/pydoc-markdown.yml b/pydoc-markdown.yml new file mode 100644 index 00000000..3c7ec4b9 --- /dev/null +++ b/pydoc-markdown.yml @@ -0,0 +1,26 @@ +# pydoc-markdown.yml – works with 4.8.2 +loaders: + - type: python + search_path: ["./"] + packages: ["aixplain"] + +renderer: + type: docusaurus + docs_base_path: docs + relative_output_path: api-reference/python + relative_sidebar_path: api_sidebar.js + sidebar_top_level_label: "Python API reference" + markdown: + source_linker: + type: github + repo: aixplain/aiXplain + use_branch: true + header_level_by_type: + Module: 2 + Class: 3 + Function: 4 + Method: 4 + +hooks: + post-render: + - python3 post_process_docs.py diff --git a/tests/functional/team_agent/inspector_functional_test.py b/tests/functional/team_agent/inspector_functional_test.py index e1679dde..73536136 100644 --- a/tests/functional/team_agent/inspector_functional_test.py +++ b/tests/functional/team_agent/inspector_functional_test.py @@ -1,5 +1,7 @@ """ Functional tests for team agents with inspectors. + +WARNING: This feature is currently in private beta. """ from dotenv import load_dotenv