-
Notifications
You must be signed in to change notification settings - Fork 5
feat: Add kernel language model service #586
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
6000e82
feat(kernel-language-model-service): Create package from template
grypez 2e7910a
feat(kernel-language-model-service): Add OllamaNodejsLanguageModelSer…
grypez e83f88a
feat(kernel-language-model-service): Construct Ollama with restricted…
grypez cb03117
build(kernel-language-model-service): Link fetch type
grypez 5e6e633
feat(kernel-language-model-service): Eventualize LanguageModel interface
grypez 65777e9
remove archetype abstraction
grypez 0a0ee43
use superstruct parsing
grypez 795493a
test(kernel-language-model-service): setup (mock) endoify
grypez 8412e8a
feat(kernel-language-model-service): Apply type suggestions
grypez 12890dc
test(kernel-language-model-service): Improve e2e coverage
grypez 4f69c0f
shorten names
grypez 2ede7de
feat(kernel-language-model-service): Improve restricted fetch explici…
grypez 67d092a
docs(kernel-language-model-service): Audit docstrings
grypez 065a442
docs(kernel-language-model-service): Update README
grypez 01c2854
fix fetch default argument
grypez File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # Changelog | ||
|
|
||
| All notable changes to this project will be documented in this file. | ||
|
|
||
| The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), | ||
| and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). | ||
|
|
||
| ## [Unreleased] | ||
|
|
||
| [Unreleased]: https://github.com/MetaMask/ocap-kernel/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| # `@ocap/kernel-language-model-service` | ||
|
|
||
| A package providing language model service implementations for the ocap kernel. This package defines interfaces and implementations for integrating various language model providers (like Ollama) into the kernel's object capability system. | ||
|
|
||
| ## Overview | ||
|
|
||
| This package provides: | ||
|
|
||
| - **Generic interfaces** for language model services that can be implemented by different providers | ||
| - **Ollama integration** for local language model inference | ||
| - **Object capability security** through hardened instances and endowment patterns | ||
| - **Type-safe configuration** using Superstruct validation | ||
|
|
||
| ## Architecture | ||
|
|
||
| The package follows the object capability pattern with clear separation of concerns: | ||
|
|
||
| - `LanguageModelService` - Factory interface for creating model instances | ||
| - `LanguageModel` - Interface for individual model instances | ||
| - Provider-specific implementations (e.g., `OllamaNodejsService`) | ||
|
|
||
| All model instances are hardened using `harden()` from `@endo/ses` for security. | ||
|
|
||
| ## Installation | ||
|
|
||
| `yarn add @ocap/kernel-language-model-service` | ||
|
|
||
| or | ||
|
|
||
| `npm install @ocap/kernel-language-model-service` | ||
|
|
||
| ## Usage | ||
|
|
||
| ### Basic Ollama Integration | ||
|
|
||
| ```typescript | ||
| import { OllamaNodejsService } from '@ocap/kernel-language-model-service/ollama/nodejs'; | ||
|
|
||
| // Create a service instance with required endowments | ||
| const service = new OllamaNodejsService({ | ||
| endowments: { fetch: global.fetch }, | ||
| }); | ||
|
|
||
| // Create a model instance | ||
| const model = await service.makeInstance({ | ||
| model: 'llama2', | ||
| options: { temperature: 0.7 }, | ||
| }); | ||
|
|
||
| // (Optional) Load the model into memory | ||
| await model.load(); | ||
|
|
||
| // Generate a response | ||
| const response = await model.sample('Hello, world!'); | ||
| for await (const chunk of response) { | ||
| console.log(chunk.response); | ||
| } | ||
|
|
||
| // (Optional) Unload the model when done | ||
| await model.unload(); | ||
| ``` | ||
|
|
||
| ### Using Host-Restricted Fetch | ||
|
|
||
| For enhanced security, you can use the host-restricted fetch utility: | ||
|
|
||
| ```typescript | ||
| import { makeHostRestrictedFetch } from '@ocap/kernel-language-model-service/ollama/fetch'; | ||
|
|
||
| const restrictedFetch = makeHostRestrictedFetch( | ||
| ['localhost:11434'], | ||
| global.fetch, | ||
| ); | ||
|
|
||
| const service = new OllamaNodejsService({ | ||
| endowments: { fetch: restrictedFetch }, | ||
| }); | ||
| ``` | ||
|
|
||
| ### Listing Available Models | ||
|
|
||
| ```typescript | ||
| const models = await service.getModels(); | ||
| console.log( | ||
| 'Available models:', | ||
| models.models.map((m) => m.name), | ||
| ); | ||
| ``` | ||
|
|
||
| ## Security Considerations | ||
|
|
||
| - **Object Capabilities**: All model instances are hardened and can be safely passed between vats | ||
| - **Endowment Pattern**: External dependencies (like `fetch`) must be explicitly provided | ||
| - **Host Restrictions**: Use `makeHostRestrictedFetch` to limit network access | ||
| - **Validation**: All configurations are validated using Superstruct schemas | ||
|
|
||
| ## API Reference | ||
|
|
||
| ### Core Types | ||
|
|
||
| - `LanguageModelService<Config, Options, Response>` - Factory for creating model instances | ||
| - `LanguageModel<Options, Response>` - Interface for model instances | ||
| - `ModelInfo<Options>` - Configuration information for a model | ||
| - `InstanceConfig<Options>` - Configuration for creating model instances | ||
|
|
||
| ### Ollama Types | ||
|
|
||
| - `OllamaNodejsService` - Node.js implementation of Ollama service | ||
| - `OllamaModelOptions` - Valid options for Ollama model generation | ||
| - `OllamaInstanceConfig` - Configuration for Ollama model instances | ||
|
|
||
| ## Contributing | ||
|
|
||
| This package is part of a monorepo. Instructions for contributing can be found in the [monorepo README](https://github.com/MetaMask/ocap-kernel#readme). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| { | ||
| "name": "@ocap/kernel-language-model-service", | ||
| "version": "0.0.0", | ||
| "private": true, | ||
| "description": "A place for implementations providing language model services to the ocap kernel", | ||
| "homepage": "https://github.com/MetaMask/ocap-kernel/tree/main/packages/kernel-language-model-service#readme", | ||
| "bugs": { | ||
| "url": "https://github.com/MetaMask/ocap-kernel/issues" | ||
| }, | ||
| "repository": { | ||
| "type": "git", | ||
| "url": "https://github.com/MetaMask/ocap-kernel.git" | ||
| }, | ||
| "type": "module", | ||
| "exports": { | ||
| "./ollama/nodejs": { | ||
| "import": { | ||
| "types": "./dist/ollama/nodejs.d.mts", | ||
| "default": "./dist/ollama/nodejs.mjs" | ||
| }, | ||
| "require": { | ||
| "types": "./dist/ollama/nodejs.d.cts", | ||
| "default": "./dist/ollama/nodejs.cjs" | ||
| } | ||
| }, | ||
| "./package.json": "./package.json" | ||
| }, | ||
| "files": [ | ||
| "dist/" | ||
| ], | ||
| "scripts": { | ||
| "build": "ts-bridge --project tsconfig.build.json --clean", | ||
| "build:docs": "typedoc", | ||
| "changelog:validate": "../../scripts/validate-changelog.sh @ocap/kernel-language-model-service", | ||
| "clean": "rimraf --glob './*.tsbuildinfo' ./.eslintcache ./coverage ./dist", | ||
| "lint": "yarn lint:eslint && yarn lint:misc --check && yarn constraints && yarn lint:dependencies", | ||
| "lint:dependencies": "depcheck", | ||
| "lint:eslint": "eslint . --cache", | ||
| "lint:fix": "yarn lint:eslint --fix && yarn lint:misc --write && yarn constraints --fix && yarn lint:dependencies", | ||
| "lint:misc": "prettier --no-error-on-unmatched-pattern '**/*.json' '**/*.md' '**/*.html' '!**/CHANGELOG.old.md' '**/*.yml' '!.yarnrc.yml' '!merged-packages/**' --ignore-path ../../.gitignore", | ||
| "publish:preview": "yarn npm publish --tag preview", | ||
| "test": "vitest run --config vitest.config.ts", | ||
| "test:e2e": "vitest run --config vitest.config.e2e.ts", | ||
| "test:clean": "yarn test --no-cache --coverage.clean", | ||
| "test:dev": "yarn test --mode development", | ||
| "test:verbose": "yarn test --reporter verbose", | ||
| "test:watch": "vitest --config vitest.config.ts" | ||
| }, | ||
| "devDependencies": { | ||
| "@arethetypeswrong/cli": "^0.17.4", | ||
| "@metamask/auto-changelog": "^5.0.1", | ||
| "@metamask/eslint-config": "^14.0.0", | ||
| "@metamask/eslint-config-nodejs": "^14.0.0", | ||
| "@metamask/eslint-config-typescript": "^14.0.0", | ||
| "@metamask/streams": "workspace:^", | ||
| "@ocap/test-utils": "workspace:^", | ||
| "@ts-bridge/cli": "^0.6.3", | ||
| "@ts-bridge/shims": "^0.1.1", | ||
| "@types/chrome": "^0.0.313", | ||
| "@typescript-eslint/eslint-plugin": "^8.29.0", | ||
| "@typescript-eslint/parser": "^8.29.0", | ||
| "@typescript-eslint/utils": "^8.29.0", | ||
| "@vitest/eslint-plugin": "^1.1.44", | ||
| "depcheck": "^1.4.7", | ||
| "eslint": "^9.23.0", | ||
| "eslint-config-prettier": "^10.1.1", | ||
| "eslint-import-resolver-typescript": "^4.3.1", | ||
| "eslint-plugin-import-x": "^4.10.0", | ||
| "eslint-plugin-jsdoc": "^50.6.9", | ||
| "eslint-plugin-n": "^17.17.0", | ||
| "eslint-plugin-prettier": "^5.2.6", | ||
| "eslint-plugin-promise": "^7.2.1", | ||
| "prettier": "^3.5.3", | ||
| "rimraf": "^6.0.1", | ||
| "typedoc": "^0.28.1", | ||
| "typescript": "~5.8.2", | ||
| "typescript-eslint": "^8.29.0", | ||
| "vite": "^6.3.5", | ||
| "vitest": "^3.2.4" | ||
| }, | ||
| "engines": { | ||
| "node": "^20 || >=22" | ||
| }, | ||
| "dependencies": { | ||
| "@metamask/superstruct": "^3.2.1", | ||
| "ollama": "^0.5.16", | ||
| "ses": "^1.13.0" | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.