Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions packages/kernel-language-model-service/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

[Unreleased]: https://github.com/MetaMask/ocap-kernel/
114 changes: 114 additions & 0 deletions packages/kernel-language-model-service/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# `@ocap/kernel-language-model-service`

A package providing language model service implementations for the ocap kernel. This package defines interfaces and implementations for integrating various language model providers (like Ollama) into the kernel's object capability system.

## Overview

This package provides:

- **Generic interfaces** for language model services that can be implemented by different providers
- **Ollama integration** for local language model inference
- **Object capability security** through hardened instances and endowment patterns
- **Type-safe configuration** using Superstruct validation

## Architecture

The package follows the object capability pattern with clear separation of concerns:

- `LanguageModelService` - Factory interface for creating model instances
- `LanguageModel` - Interface for individual model instances
- Provider-specific implementations (e.g., `OllamaNodejsService`)

All model instances are hardened using `harden()` from `@endo/ses` for security.

## Installation

`yarn add @ocap/kernel-language-model-service`

or

`npm install @ocap/kernel-language-model-service`

## Usage

### Basic Ollama Integration

```typescript
import { OllamaNodejsService } from '@ocap/kernel-language-model-service/ollama/nodejs';

// Create a service instance with required endowments
const service = new OllamaNodejsService({
endowments: { fetch: global.fetch },
});

// Create a model instance
const model = await service.makeInstance({
model: 'llama2',
options: { temperature: 0.7 },
});

// (Optional) Load the model into memory
await model.load();

// Generate a response
const response = await model.sample('Hello, world!');
for await (const chunk of response) {
console.log(chunk.response);
}

// (Optional) Unload the model when done
await model.unload();
```

### Using Host-Restricted Fetch

For enhanced security, you can use the host-restricted fetch utility:

```typescript
import { makeHostRestrictedFetch } from '@ocap/kernel-language-model-service/ollama/fetch';

const restrictedFetch = makeHostRestrictedFetch(
['localhost:11434'],
global.fetch,
);

const service = new OllamaNodejsService({
endowments: { fetch: restrictedFetch },
});
```

### Listing Available Models

```typescript
const models = await service.getModels();
console.log(
'Available models:',
models.models.map((m) => m.name),
);
```

## Security Considerations

- **Object Capabilities**: All model instances are hardened and can be safely passed between vats
- **Endowment Pattern**: External dependencies (like `fetch`) must be explicitly provided
- **Host Restrictions**: Use `makeHostRestrictedFetch` to limit network access
- **Validation**: All configurations are validated using Superstruct schemas

## API Reference

### Core Types

- `LanguageModelService<Config, Options, Response>` - Factory for creating model instances
- `LanguageModel<Options, Response>` - Interface for model instances
- `ModelInfo<Options>` - Configuration information for a model
- `InstanceConfig<Options>` - Configuration for creating model instances

### Ollama Types

- `OllamaNodejsService` - Node.js implementation of Ollama service
- `OllamaModelOptions` - Valid options for Ollama model generation
- `OllamaInstanceConfig` - Configuration for Ollama model instances

## Contributing

This package is part of a monorepo. Instructions for contributing can be found in the [monorepo README](https://github.com/MetaMask/ocap-kernel#readme).
89 changes: 89 additions & 0 deletions packages/kernel-language-model-service/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
{
"name": "@ocap/kernel-language-model-service",
Comment thread
grypez marked this conversation as resolved.
"version": "0.0.0",
"private": true,
"description": "A place for implementations providing language model services to the ocap kernel",
"homepage": "https://github.com/MetaMask/ocap-kernel/tree/main/packages/kernel-language-model-service#readme",
"bugs": {
"url": "https://github.com/MetaMask/ocap-kernel/issues"
},
"repository": {
"type": "git",
"url": "https://github.com/MetaMask/ocap-kernel.git"
},
"type": "module",
"exports": {
"./ollama/nodejs": {
"import": {
"types": "./dist/ollama/nodejs.d.mts",
"default": "./dist/ollama/nodejs.mjs"
},
"require": {
"types": "./dist/ollama/nodejs.d.cts",
"default": "./dist/ollama/nodejs.cjs"
}
},
"./package.json": "./package.json"
},
"files": [
"dist/"
],
"scripts": {
"build": "ts-bridge --project tsconfig.build.json --clean",
"build:docs": "typedoc",
"changelog:validate": "../../scripts/validate-changelog.sh @ocap/kernel-language-model-service",
"clean": "rimraf --glob './*.tsbuildinfo' ./.eslintcache ./coverage ./dist",
"lint": "yarn lint:eslint && yarn lint:misc --check && yarn constraints && yarn lint:dependencies",
"lint:dependencies": "depcheck",
"lint:eslint": "eslint . --cache",
"lint:fix": "yarn lint:eslint --fix && yarn lint:misc --write && yarn constraints --fix && yarn lint:dependencies",
"lint:misc": "prettier --no-error-on-unmatched-pattern '**/*.json' '**/*.md' '**/*.html' '!**/CHANGELOG.old.md' '**/*.yml' '!.yarnrc.yml' '!merged-packages/**' --ignore-path ../../.gitignore",
"publish:preview": "yarn npm publish --tag preview",
"test": "vitest run --config vitest.config.ts",
"test:e2e": "vitest run --config vitest.config.e2e.ts",
"test:clean": "yarn test --no-cache --coverage.clean",
"test:dev": "yarn test --mode development",
"test:verbose": "yarn test --reporter verbose",
"test:watch": "vitest --config vitest.config.ts"
},
"devDependencies": {
"@arethetypeswrong/cli": "^0.17.4",
"@metamask/auto-changelog": "^5.0.1",
"@metamask/eslint-config": "^14.0.0",
"@metamask/eslint-config-nodejs": "^14.0.0",
"@metamask/eslint-config-typescript": "^14.0.0",
"@metamask/streams": "workspace:^",
"@ocap/test-utils": "workspace:^",
"@ts-bridge/cli": "^0.6.3",
"@ts-bridge/shims": "^0.1.1",
"@types/chrome": "^0.0.313",
"@typescript-eslint/eslint-plugin": "^8.29.0",
"@typescript-eslint/parser": "^8.29.0",
"@typescript-eslint/utils": "^8.29.0",
"@vitest/eslint-plugin": "^1.1.44",
"depcheck": "^1.4.7",
"eslint": "^9.23.0",
"eslint-config-prettier": "^10.1.1",
"eslint-import-resolver-typescript": "^4.3.1",
"eslint-plugin-import-x": "^4.10.0",
"eslint-plugin-jsdoc": "^50.6.9",
"eslint-plugin-n": "^17.17.0",
"eslint-plugin-prettier": "^5.2.6",
"eslint-plugin-promise": "^7.2.1",
"prettier": "^3.5.3",
"rimraf": "^6.0.1",
"typedoc": "^0.28.1",
"typescript": "~5.8.2",
"typescript-eslint": "^8.29.0",
"vite": "^6.3.5",
"vitest": "^3.2.4"
},
"engines": {
"node": "^20 || >=22"
},
"dependencies": {
"@metamask/superstruct": "^3.2.1",
"ollama": "^0.5.16",
"ses": "^1.13.0"
}
}
Loading
Loading