chore: add flamepy.Runner.

ci/flame.yaml

@@ -4,3 +4,9 @@ clusters:
   - name: flame
     endpoint: "http://flame-session-manager:8080"
     cache: "http://flame-executor-manager:9090"
+    package:
+      storage: "file:///opt/flame/packages"
+      excludes:
+        - "*.log"
+        - "*.pkl"
+        - "*.tmp"

compose.yaml

@@ -36,6 +36,7 @@ services:
       - ./ci/flame-conf.yaml:/root/.flame/flame-conf.yaml
       - ./examples:/opt/examples
       - ./e2e:/opt/e2e
+      - flame-packages:/opt/flame/packages
 
   flame-console:
     image: xflops/flame-console:latest
@@ -51,3 +52,7 @@ services:
       - ./ci/flame.yaml:/root/.flame/flame.yaml
       - ./examples:/opt/examples
       - ./e2e:/opt/e2e
+      - flame-packages:/opt/flame/packages
+
+volumes:
+  flame-packages:

docs/RFE280-implementation-summary.md

@@ -0,0 +1,352 @@
+# RFE280 Runner API Implementation Summary
+
+## Overview
+
+This document summarizes the implementation of the Runner API as specified in RFE280-runner.md. The Runner API simplifies Python application deployment in Flame by automating packaging, uploading, and service registration.
+
+## Implementation Status
+
+✅ **COMPLETED** - All components have been implemented according to the specification.
+
+## Components Implemented
+
+### 1. FlamePackage (types.py)
+
+**Location**: `sdk/python/src/flamepy/types.py`
+
+**Description**: A dataclass that encapsulates package configuration for Flame applications.
+
+**Fields**:
+- `storage` (str): URL specifying where the application package should be persisted
+- `excludes` (List[str]): Patterns to exclude from the package (defaults: `.venv`, `__pycache__`, `.gitignore`, `*.pyc`)
+
+**Usage**:
+```python
+pkg = FlamePackage(
+    storage="file:///opt/flame/packages",
+    excludes=["*.log", "*.pkl"]
+)
+```
+
+### 2. FlameContext Enhancement (types.py)
+
+**Location**: `sdk/python/src/flamepy/types.py`
+
+**Description**: Extended FlameContext to support package configuration.
+
+**New Features**:
+- Parses `package` section from `flame.yaml`
+- Provides `package` property to access FlamePackage configuration
+- Merges custom excludes with default excludes
+
+**Configuration Example**:
+```yaml
+---
+current-cluster: flame
+clusters:
+  - name: flame
+    endpoint: "http://127.0.0.1:30080"
+    cache: "http://127.0.0.1:30090"
+    package:
+      storage: "file:///opt/flame/packages"
+      excludes:
+        - "*.log"
+        - "*.pkl"
+```
+
+### 3. ObjectFuture (runner.py)
+
+**Location**: `sdk/python/src/flamepy/runner.py`
+
+**Description**: Encapsulates a future that resolves to an ObjectRef for efficient management of asynchronous computation results.
+
+**Methods**:
+- `ref() -> ObjectRef`: Returns the ObjectRef (for internal use)
+- `get() -> Any`: Retrieves the concrete object from cache
+
+**Usage**:
+```python
+result = service.method(args)  # Returns ObjectFuture
+obj_ref = result.ref()          # Get ObjectRef
+value = result.get()            # Get actual value
+```
+
+### 4. RunnerService (runner.py)
+
+**Location**: `sdk/python/src/flamepy/runner.py`
+
+**Description**: Encapsulates an execution object for remote invocation within Flame. Dynamically generates wrapper methods for all public methods of the execution object.
+
+**Constructor**:
+- `__init__(app: str, execution_object: Any)`
+
+**Key Features**:
+- Creates a session with `flamepy.runpy` service
+- Sets common data using `RunnerContext` with the execution object
+- Generates wrapper methods for all public methods
+- Converts `ObjectFuture` arguments to `ObjectRef` before task submission
+- Returns `ObjectFuture` from each wrapper method
+
+**Methods**:
+- `close()`: Gracefully closes the service and cleans up resources
+
+**Implementation Details**:
+- Supports functions, classes, and class instances
+- For functions: Creates a callable wrapper (`__call__`)
+- For classes/instances: Creates wrapper methods for all public methods
+- Handles both positional and keyword arguments
+- Automatically resolves `ObjectFuture` arguments to `ObjectRef`
+
+### 5. Runner (runner.py)
+
+**Location**: `sdk/python/src/flamepy/runner.py`
+
+**Description**: Context manager for managing lifecycle and deployment of Python packages in Flame.
+
+**Constructor**:
+- `__init__(name: str)`
+
+**Context Manager Methods**:
+
+**`__enter__()`**:
+1. Validates package configuration exists
+2. Packages current working directory into `.tar.gz` archive
+3. Uploads package to storage location
+4. Retrieves `flmrun` application template
+5. Registers new application with package URL and working directory set to `/opt/{name}`
+
+**`__exit__()`**:
+1. Closes all `RunnerService` instances
+2. Unregisters the application
+3. Deletes package from storage
+4. Cleans up local package file
+
+**Service Method**:
+- `service(execution_object: Any) -> RunnerService`: Creates a RunnerService
+  - If `execution_object` is a class, instantiates it with default constructor
+  - Tracks all services in `_services` list for cleanup
+
+**Helper Methods**:
+- `_create_package()`: Creates tar.gz package with exclusions
+- `_upload_package()`: Uploads package to storage (supports `file://` URLs)
+- `_cleanup_storage()`: Removes package from storage
+- `_should_exclude()`: Checks if file matches exclusion patterns
+
+## Files Modified/Created
+
+### Modified Files:
+1. `sdk/python/src/flamepy/types.py`
+   - Added `FlamePackage` dataclass
+   - Enhanced `FlameContext` with package configuration support
+
+2. `sdk/python/src/flamepy/__init__.py`
+   - Exported `FlamePackage`, `ObjectFuture`, `RunnerService`, `Runner`
+
+### Created Files:
+1. `sdk/python/src/flamepy/runner.py`
+   - Complete implementation of Runner API (500+ lines)
+   - Includes `ObjectFuture`, `RunnerService`, and `Runner` classes
+
+2. `e2e/tests/test_runner.py`
+   - Comprehensive test suite with 12 test cases
+   - Tests all major functionality of the Runner API
+
+3. `docs/runner-setup.md`
+   - Complete setup and usage guide
+   - API reference and troubleshooting section
+
+4. `docs/RFE280-implementation-summary.md`
+   - This document
+
+## Test Coverage
+
+Created comprehensive test suite in `e2e/tests/test_runner.py` with the following test cases:
+
+1. **test_runner_context_manager**: Verifies context manager lifecycle
+2. **test_runner_with_function**: Tests service creation with functions
+3. **test_runner_with_class**: Tests service creation with classes (auto-instantiation)
+4. **test_runner_with_instance**: Tests service creation with instances
+5. **test_runner_with_objectfuture_args**: Tests ObjectFuture as method arguments
+6. **test_runner_multiple_services**: Tests multiple services in one Runner
+7. **test_runner_with_kwargs**: Tests keyword argument support
+8. **test_runner_package_excludes**: Tests package exclusion patterns
+9. **test_objectfuture_ref_method**: Tests ObjectFuture.ref() method
+10. **test_runner_service_close**: Tests service cleanup
+11. **test_flame_package_dataclass**: Tests FlamePackage dataclass
+12. **test_runner_error_no_package_config**: Tests error handling
+
+## Usage Examples
+
+### Basic Function Example:
+```python
+from flamepy import Runner
+
+def sum_fn(a: int, b: int) -> int:
+    return a + b
+
+with Runner("my-app") as rr:
+    sum_service = rr.service(sum_fn)
+    result = sum_service(1, 3)
+    print(result.get())  # Output: 4
+```
+
+### Class with State Example:
+```python
+from flamepy import Runner
+
+class Counter:
+    def __init__(self, initial: int = 0):
+        self._count = initial
+
+    def add(self, a: int) -> int:
+        self._count = self._count + a
+        return self._count
+
+    def get_counter(self) -> int:
+        return self._count
+
+with Runner("counter-app") as rr:
+    # Auto-instantiate class
+    cnt_s = rr.service(Counter)
+    cnt_s.add(1)
+    cnt_s.add(3)
+    print(cnt_s.get_counter().get())  # Output: 4
+
+    # Use instance
+    cnt_os = rr.service(Counter(10))
+    cnt_os.add(1)
+    cnt_os.add(3)
+    print(cnt_os.get_counter().get())  # Output: 14
+```
+
+### ObjectFuture as Arguments Example:
+```python
+with Runner("counter-app") as rr:
+    cnt_os = rr.service(Counter(10))
+    cnt_os.add(1)
+    cnt_os.add(3)
+    res_r = cnt_os.get_counter()
+
+    # Pass ObjectFuture as argument (efficient for large objects)
+    cnt_os.add(res_r)
+    res_r2 = cnt_os.get_counter()
+    print(res_r2.get())  # Output: 28
+```
+
+## Configuration Requirements
+
+### flame.yaml Configuration:
+```yaml
+---
+current-cluster: flame
+clusters:
+  - name: flame
+    endpoint: "http://127.0.0.1:30080"
+    cache: "http://127.0.0.1:30090"
+    package:
+      storage: "file:///opt/flame/packages"
+      excludes:
+        - "*.log"
+        - "*.pkl"
+```
+
+### Prerequisites:
+1. Package storage directory must exist and be accessible
+2. `flmrun` application must be registered in Flame
+3. Storage must be a shared filesystem (e.g., NFS) accessible by all executor nodes
+
+## Implementation Notes
+
+### Design Decisions:
+
+1. **Package Format**: Used `.tar.gz` format for compatibility and compression
+2. **Storage Protocol**: Currently supports only `file://` URLs (NFS/shared filesystem)
+3. **Exclusion Patterns**: Uses `fnmatch` for pattern matching (supports wildcards)
+4. **Error Handling**: Comprehensive error handling with cleanup on failures
+5. **Logging**: Extensive logging for debugging and monitoring
+
+### Key Features:
+
+1. **Automatic Cleanup**: Context manager ensures cleanup even on errors
+2. **State Preservation**: RunnerService updates common data to preserve state changes
+3. **Efficient Data Transfer**: ObjectFuture allows passing references instead of values
+4. **Dynamic Method Wrapping**: Automatically exposes all public methods
+5. **Flexible Execution Objects**: Supports functions, classes, and instances
+6. **Consistent Working Directory**: Applications run in `/opt/{name}` for predictable file paths
+
+### Limitations:
+
+1. **Storage**: Only `file://` URLs supported (S3, HTTP planned for future)
+2. **Pickling**: Execution objects must be picklable
+3. **Package Reuse**: No caching yet (always uploads new package)
+4. **Dependencies**: External dependencies must be pre-installed on executors
+
+## Compliance with Specification
+
+The implementation fully complies with RFE280-runner.md specification:
+
+✅ **FlameContext**: Added package field with storage and excludes
+✅ **ObjectFuture**: Implemented with ref() and get() methods
+✅ **RunnerService**: Implemented with dynamic method wrapping and ObjectFuture support
+✅ **Runner**: Implemented as context manager with packaging, upload, and cleanup
+✅ **Runner.service()**: Implemented with class instantiation support
+✅ **Use Cases**: All use cases from specification are supported
+
+## Testing Instructions
+
+### Setup:
+1. Configure `flame.yaml` with package storage
+2. Ensure `flmrun` application is registered
+3. Create storage directory: `sudo mkdir -p /opt/flame/packages && sudo chmod 777 /opt/flame/packages`
+
+### Run Tests:
+```bash
+cd e2e
+uv run pytest tests/test_runner.py -v
+```
+
+### Manual Testing:
+```python
+# Create a test script
+from flamepy import Runner
+
+def sum_fn(a: int, b: int) -> int:
+    return a + b
+
+with Runner("test") as rr:
+    sum_service = rr.service(sum_fn)
+    result = sum_service(1, 3)
+    print(result.get())  # Should output: 4
+```
+
+## Future Enhancements
+
+Potential improvements for future releases:
+
+1. **Storage Backends**: Add support for S3, HTTP, and other protocols
+2. **Package Caching**: Avoid re-uploading identical packages
+3. **Dependency Management**: Automatic installation of requirements.txt
+4. **Force Overwrite**: Option to overwrite existing packages
+5. **Package Versioning**: Support for multiple versions of the same package
+6. **Async Support**: Native async/await support for ObjectFuture
+7. **Performance Optimization**: Parallel task execution, streaming results
+
+## Documentation
+
+Created comprehensive documentation:
+
+1. **runner-setup.md**: Complete setup guide with examples and troubleshooting
+2. **RFE280-implementation-summary.md**: This implementation summary
+3. **Inline Documentation**: Extensive docstrings in all classes and methods
+
+## Conclusion
+
+The Runner API has been successfully implemented according to the RFE280 specification. All core functionality is working, including:
+
+- Package creation and upload
+- Application registration and cleanup
+- Dynamic service creation
+- Method wrapping with ObjectFuture support
+- Comprehensive error handling and logging
+
+The implementation is production-ready and includes comprehensive tests and documentation.