Add doc improve some code

.github/workflows/docs.yml

@@ -0,0 +1,45 @@
+name: Publish Docs to GitHub Pages
+
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - 'docs/**'
+      - 'mkdocs.yml'
+      - 'src/**'
+      - '.github/workflows/docs.yml'
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0       # full history needed for git-committers plugin
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+          cache: pip
+
+      - name: Install doc dependencies
+        run: |
+          pip install \
+            mkdocs-material \
+            mkdocstrings[python] \
+            mkdocs-git-revision-date-localized-plugin \
+            griffe
+
+      - name: Install claimed package (for mkdocstrings introspection)
+        run: pip install -e .
+
+      - name: Build & deploy docs
+        run: mkdocs gh-deploy --force --clean --verbose

.gitignore

@@ -12,3 +12,4 @@ dist
 *.bak
 *.swp
 .DS_Store
+site/

docs/c3/create-gridwrapper.md

@@ -0,0 +1,50 @@
+# create_gridwrapper
+
+Wraps an existing component to run in parallel over a collection of inputs
+using one of several storage backends.
+
+## CLI
+
+```bash
+c3_create_gridwrapper <source_file> [options]
+```
+
+| Option | Type | Default | Description |
+|---|---|---|---|
+| `source_file` | path | *required* | `.ipynb` or `.py` component to wrap |
+| `--backend` | str | `local` | Storage backend (see table below) |
+| `--component-inputs` | str | `''` | Comma-separated parameter names that vary per grid cell |
+| `--component-dependencies` | str | `''` | Pip dependencies to inject |
+| `--repository` | str | | Container registry namespace |
+| `--log-level` | str | `WARNING` | Python logging level |
+
+## Backends
+
+| Key | Description |
+|---|---|
+| `local` | Local filesystem, simple parallelism |
+| `cos` | IBM COS – iterate over objects in a bucket prefix |
+| `s3kv` | MLX S3 key-value store backend |
+| `simple_grid_wrapper` | Source-only, minimal overhead |
+| `folder_grid_wrapper` | Separate source and target folder |
+| `legacy_cos_grid_wrapper` | Older COS format |
+
+## Python API
+
+::: claimed.c3.create_gridwrapper
+    options:
+      members:
+        - wrap_component
+        - create_gridwrapper
+
+## Example
+
+```bash
+# Wrap a training script to process every CSV in a COS bucket in parallel
+c3_create_gridwrapper train_model.py \
+    --backend cos \
+    --component-inputs input_file \
+    --repository docker.io/myuser
+```
+
+This emits `gw_train_model.py` which, when containerised, launches one worker per input file.

docs/c3/create-operator.md

@@ -0,0 +1,38 @@
+# create_operator
+
+Builds a Docker image and generates KFP, CWL, and Kubernetes descriptors
+from a Jupyter notebook, Python script, or R script.
+
+## CLI
+
+```bash
+c3_create_operator <source_file> [options]
+```
+
+| Option | Type | Default | Description |
+|---|---|---|---|
+| `source_file` | path | *required* | `.ipynb`, `.py`, or `.R` file |
+| `--repository` | str | *required* | Container registry namespace, e.g. `docker.io/myuser` |
+| `--version` | str | auto | Image tag; auto-detected from `image_version` variable in source |
+| `--additional-files` | list | `[]` | Extra files to `ADD` into the image |
+| `--dockerfile` | path | auto | Custom Dockerfile template |
+| `--log-level` | str | `WARNING` | Python logging level |
+
+## Python API
+
+::: claimed.c3.create_operator
+    options:
+      members:
+        - create_operator
+        - create_dockerfile
+
+## Output Files
+
+After a successful run you will find:
+
+| File | Description |
+|---|---|
+| `<name>.dockerfile` | Generated Dockerfile |
+| `<name>.yaml` | KubeFlow Pipelines component spec |
+| `<name>.job.yaml` | Kubernetes Job spec |
+| `<name>.cwl` | CWL component descriptor |

docs/c3/index.md

@@ -0,0 +1,61 @@
+# C3 – CLAIMED Component Compiler
+
+C3 automates the transformation of arbitrary code assets into fully portable, executable AI components.
+
+---
+
+## What C3 does
+
+```
+ ┌──────────────────────┐
+ │  .ipynb / .py / .R   │   ← your code
+ └──────────┬───────────┘
+            │  c3_create_operator
+            ▼
+ ┌──────────────────────────────────────────┐
+ │  Dockerfile           (build + push)     │
+ │  KubeFlow component YAML                 │
+ │  Kubernetes Job YAML                     │
+ │  CWL component descriptor                │
+ └──────────────────────────────────────────┘
+```
+
+C3 reads **parameter declarations** from the top of your source file:
+
+```python
+import os
+
+# description of my_param
+my_param = os.environ.get('my_param', 'default_value')
+```
+
+Each `os.environ.get(...)` line is parsed into a typed, documented parameter
+that appears in the generated YAML descriptors and KFP UI.
+
+---
+
+## Modules
+
+| Module | CLI entry-point | Purpose |
+|---|---|---|
+| [`create_operator`](create-operator.md) | `c3_create_operator` | Build container images and component descriptors |
+| [`create_gridwrapper`](create-gridwrapper.md) | `c3_create_gridwrapper` | Wrap a component for parallel grid execution |
+| [`create_containerless_operator`](create-operator.md) | `c3_create_containerless_operator` | Containerless variant (runs in-process) |
+| [`operator_utils`](operator-utils.md) | – | Shared helpers (connection strings, logging) |
+| `parser` | – | Source-file parameter parser |
+| `notebook` | – | Jupyter notebook handler |
+| `pythonscript` | – | Python script handler |
+| `rscript` | – | R script handler |
+
+---
+
+## Grid Compute Backends
+
+| Backend key | Description |
+|---|---|
+| `local` | Plain local filesystem |
+| `cos` / `cos_grid_wrapper` | IBM Cloud Object Storage |
+| `s3kv` | S3-backed key-value store (MLX) |
+| `simple_grid_wrapper` | Minimal wrapper – source folder only |
+| `folder_grid_wrapper` | Source **and** target folder variant |
+| `legacy_cos_grid_wrapper` | Older COS format, kept for backwards compatibility |

docs/c3/operator-utils.md

@@ -0,0 +1,39 @@
+# operator_utils
+
+Shared utility helpers used across C3 and the component library.
+
+## Python API
+
+::: claimed.c3.operator_utils
+
+## Connection String Format
+
+Many CLAIMED components accept a `cos_connection` parameter in the following URI format:
+
+```
+[cos|s3]://access_key_id:secret_access_key@endpoint_host/bucket/path
+```
+
+**Examples:**
+
+```
+s3://AKIAIOSFODNN7EXAMPLE:wJalrXUtnFEMI@s3.us-east-1.amazonaws.com/my-bucket/data/
+cos://mykey:mysecret@s3.eu-de.cloud-object-storage.appdomain.cloud/my-bucket/models/
+```
+
+### `explode_connection_string(cs)`
+
+Parses the URI into its components:
+
+```python
+from claimed.c3.operator_utils import explode_connection_string
+
+access_key_id, secret_access_key, endpoint, path = explode_connection_string(
+    's3://KEY:SECRET@s3.eu-de.cloud-object-storage.appdomain.cloud/my-bucket/prefix'
+)
+# endpoint → 'https://s3.eu-de.cloud-object-storage.appdomain.cloud'
+# path     → 'my-bucket/prefix'
+```
+
+If the string does not start with `cos://` or `s3://`, the input is returned as-is in the `path` field
+(useful when passing a plain local path or a Kubernetes secret reference).

docs/cli.md

@@ -0,0 +1,135 @@
+# CLI Reference
+
+The `claimed` command is the single entry-point for the CLAIMED framework.
+
+---
+
+## Synopsis
+
+```
+claimed <subcommand> [options]
+```
+
+---
+
+## Subcommands
+
+### `claimed run`
+
+Directly invoke the `run()` function of any CLAIMED Python module.
+
+```
+claimed run <module.path> [--param-name value ...] [--help]
+```
+
+**Arguments**
+
+| Argument | Description |
+|---|---|
+| `module.path` | Fully-qualified Python module containing a `run()` function (e.g. `claimed.components.util.cosutils`) |
+| `--<param-name>` | Any parameter accepted by `run()`. Hyphens are converted to underscores. |
+| `--help` | Print the function signature, docstring, and parameter list, then exit. |
+
+**Type coercion**
+
+String values from the command line are automatically cast to the type declared in the function signature
+(annotation or default-value type).  
+For example, `--recursive true` is cast to `bool` if the parameter is annotated as `bool`.
+
+**Examples**
+
+```bash
+# List objects in a COS bucket
+claimed run claimed.components.util.cosutils \
+    --cos-connection s3://KEY:SECRET@endpoint/bucket \
+    --operation ls \
+    --local-path .
+
+# Download a file
+claimed run claimed.components.util.cosutils \
+    --cos-connection s3://KEY:SECRET@endpoint/bucket/file.zip \
+    --operation get \
+    --local-path .
+
+# Show help for any module
+claimed run claimed.components.util.cosutils --help
+
+# CPU benchmark
+claimed run claimed.components.util.gpu_performance_test \
+    --mode cpu \
+    --matrix-size 4096 \
+    --iterations 100
+```
+
+---
+
+### `claimed create operator`
+
+Generate a container image + KFP/CWL/Kubernetes descriptors from a script or notebook.
+
+```
+claimed create operator <script_or_notebook> [options]
+```
+
+| Option | Description |
+|---|---|
+| `--repository` | Container registry namespace, e.g. `docker.io/myuser` |
+| `--version` | Image tag (default: auto-detected from script) |
+| `--additional-files` | Space-separated list of extra files to bundle |
+
+Example:
+
+```bash
+claimed create operator my_script.py --repository docker.io/myuser
+```
+
+---
+
+### `claimed create gridwrapper`
+
+Wrap a component so it executes in parallel over a collection of inputs.
+
+```
+claimed create gridwrapper <script_or_notebook> [options]
+```
+
+| Option | Description |
+|---|---|
+| `--backend` | Storage backend: `local` \| `cos` \| `s3kv` \| `simple_grid_wrapper` \| `folder_grid_wrapper` |
+| `--component-inputs` | Comma-separated parameter names that vary across grid cells |
+| `--repository` | Container registry namespace |
+
+Example:
+
+```bash
+claimed create gridwrapper my_script.py \
+    --backend cos \
+    --component-inputs input_file \
+    --repository docker.io/myuser
+```
+
+---
+
+### `claimed --component` *(legacy)*
+
+Run a component image via Docker.
+
+```
+claimed --component <image> [--param-name value ...]
+```
+
+| Option | Description |
+|---|---|
+| `--component` | Docker image reference, e.g. `docker.io/claimed/my-op:latest` |
+| `--<param-name>` | Environment variable to pass into the container |
+
+Set `CLAIMED_DATA_PATH` to mount a local directory as `/opt/app-root/src/data` inside the container.
+
+---
+
+## Environment Variables
+
+| Variable | Effect |
+|---|---|
+| `CLAIMED_DATA_PATH` | Local path mounted as `/opt/app-root/src/data` when using `--component` |
+| `CLAIMED_CONTAINERLESS_OPERATOR_PATH` | Root path for containerless operator resolution |