feat: Add multi-tenant file encryption capability#828
feat: Add multi-tenant file encryption capability#828ZaynJarvis merged 5 commits intovolcengine:mainfrom
Conversation
|
Failed to generate code suggestions for PR |
|
能否将 ov crypto 改为 ov system crypto，不要作为一级子命令 |
…at && add encryption config example
已将 ov crypto 改为 ov system crypto |
qin-ctx
left a comment
qin-ctx
left a comment
There was a problem hiding this comment.
Post-merge Review
This PR adds multi-tenant file encryption — a valuable security feature. The envelope format design and backward compatibility for unencrypted files are well done.
However, there are several correctness and security issues that should be addressed as follow-ups. See inline comments for details.
|
|
||
| # If encryption is enabled and not reading from offset, try to decrypt | ||
| if self._encryptor and offset == 0 and size == -1: | ||
| raw = await self._decrypt_content(raw, ctx=ctx) |
There was a problem hiding this comment.
[Bug] (blocking) Partial reads return ciphertext when encryption is enabled.
When offset != 0 or size != -1, decryption is skipped but AGFS returns a byte slice from the middle of the encrypted blob. The caller receives raw ciphertext instead of plaintext.
The same issue exists in read_file() (line 1562) — it reads the full file from AGFS but only decrypts when offset == 0 and limit == -1. When limit/offset are non-default, the encrypted bytes are decoded as UTF-8 and sliced by line — producing garbage.
Suggested fix: when self._encryptor is present, always read and decrypt the full file first, then apply offset/size slicing on the decrypted plaintext.
# In read():
if self._encryptor:
# Must read full file for decryption
result = self.agfs.read(path, 0, -1)
raw = ... # extract bytes
raw = await self._decrypt_content(raw, ctx=ctx)
if offset > 0 or size != -1:
raw = raw[offset:offset+size] if size != -1 else raw[offset:]
return raw| Args: | ||
| key_file: Root Key file path | ||
| """ | ||
| self.key_file = Path(key_file) |
There was a problem hiding this comment.
[Bug] (blocking) Path(key_file) does not expand ~ — key file is created at the wrong location.
Path("~/.openviking/master.key") treats ~ literally (looks for ./~/.openviking/ relative to CWD), not as the home directory. Since the Rust CLI (ov system crypto init-key) correctly resolves via dirs::home_dir(), the Python code and CLI will operate on different key files. This can lead to:
- Python creates a new random key at
./~/.openviking/master.key - Data encrypted with this accidental key cannot be decrypted by anyone who fixes the path later
Fix:
self.key_file = Path(key_file).expanduser()The same issue exists in config.py:70 (_validate_local_provider_config).
| """ | ||
| if self._root_key is None: | ||
| # Use a fixed seed for root key derivation | ||
| self._root_key = b"OpenViking_Vault_Root_Seed_Key_v1" |
There was a problem hiding this comment.
[Design] (blocking) VaultProvider.get_root_key() returns a hardcoded constant, and encrypt_file_key() ignores account_id — no per-tenant key isolation.
The PR claims "multi-tenant encryption" with "different accounts use independent encryption keys for isolation" (from issue #827). However, for VaultProvider:
get_root_key()returnsb"OpenViking_Vault_Root_Seed_Key_v1"— known to anyone who reads the sourceencrypt_file_key()encrypts all file keys with the same Vault transit key (openviking-root-key) regardless ofaccount_id
This means all tenants share a single encryption key. Decrypting one tenant's data grants access to all tenants' data.
The same issue applies to VolcengineKMSProvider (line 483).
Suggested approaches:
- Create per-account transit keys:
openviking-account-{account_id} - Or use Vault's
contextparameter for account-scoped derived keys - For KMS, use the
EncryptionContextfield with account_id
| } | ||
| } | ||
| "encryption": { | ||
| "enabled": true, |
There was a problem hiding this comment.
[Bug] (blocking) Invalid JSON — missing comma before "encryption".
The "parsers" block closes on line 164 but there is no comma before "encryption" on line 165. This makes the example file unparseable.
- }
+ },
"encryption": {| raw = content.content if hasattr(content, "content") else b"" | ||
|
|
||
| # Decrypt content if encryption is enabled | ||
| if self._viking_fs._encryptor: |
There was a problem hiding this comment.
[Design] (non-blocking) Directly accessing self._viking_fs._encryptor breaks VikingFS encapsulation.
_read_json and _write_json reach into VikingFS's private _encryptor member. If VikingFS's encryption internals change (e.g., the encryptor is moved or renamed), this code breaks silently.
Consider either:
- Exposing a public
encrypt_bytes(account_id, data)/decrypt_bytes(account_id, data)method on VikingFS - Or passing the encryptor to APIKeyManager directly at construction time
| root_key = await self.get_root_key() | ||
| return await self._hkdf_derive(root_key, account_id) | ||
|
|
||
| async def _hkdf_derive(self, root_key: bytes, account_id: str) -> bytes: |
There was a problem hiding this comment.
[Design] (non-blocking) _hkdf_derive is duplicated across all three providers with slightly different salt/info constants.
LocalFileProvider uses salt=b"openviking-kek-salt-v1", while VaultProvider and VolcengineKMSProvider use salt=b"OpenViking_KDF_Salt". This inconsistency is confusing since the method body is identical.
Consider extracting _hkdf_derive into the RootKeyProvider base class with configurable salt/info, or into a standalone utility function.
| parent_dir = key_file.parent | ||
| if not parent_dir.exists(): | ||
| try: | ||
| parent_dir.mkdir(parents=True, exist_ok=True) |
There was a problem hiding this comment.
[Suggestion] (non-blocking) Validation function has directory-creation side effects.
_validate_local_provider_config calls parent_dir.mkdir(parents=True, exist_ok=True) during validation. A validation function should be read-only — creating directories should happen during bootstrap_encryption().
| self._client = hvac.Client(url=self.addr, token=self.token) | ||
|
|
||
| # Verify Vault is accessible | ||
| if not self._client.is_authenticated(): |
There was a problem hiding this comment.
[Suggestion] (non-blocking) Synchronous hvac calls inside async methods block the event loop.
_get_client, _ensure_transit_engine_enabled, _ensure_root_key_exists make synchronous HTTP calls via hvac but are declared async. This blocks the asyncio event loop during Vault communication.
Consider wrapping synchronous calls with asyncio.to_thread() or using an async Vault client.
| key_file.write_text(root_key.hex()) | ||
| os.chmod(key_file, 0o600) | ||
|
|
||
| provider = LocalFileProvider(key_file=str(key_file)) |
There was a problem hiding this comment.
[Suggestion] (non-blocking) Integration tests only cover LocalFileProvider.
The 1334-line test file exclusively uses LocalFileProvider. VaultProvider and VolcengineKMSProvider have no integration tests. At minimum, mock-based tests should verify:
- Envelope format compatibility across providers
- Provider-type byte is correctly round-tripped
encrypt_file_key/decrypt_file_keyinteract with Vault/KMS APIs correctly
4 participants
Description
Added multi-tenant file encryption capability
Related Issue
#827
Type of Change
Changes Made
Testing
Checklist
Screenshots (if applicable)
Additional Notes
测试
测试配置
ov.conf
Local Provider
{ "encryption": { "enabled": true, "provider": "local", "local": { "key_file": "~/.openviking/master.key" } } }vault Provider
{ "encryption": { "enabled": true, "provider": "vault", "vault": { "address": "http://127.0.0.1:8200", "token": "my-vault-token" } } }volcengine KMS Provider
{ "encryption": { "enabled": true "provider": "volcengine_kms" "volc": { "key_id": "my-volc-key-id" "key_secret": "my-volc-key-secret" } } }测试执行
生成root密钥对
当不存在
~/.openviking/master.key文件时，会生成一个新的密钥.$ ov system crypto init-key Successfully generated root key at: ～/.openviking/master.key Key permissions set to 0600 (owner read/write only)当存在
~/.openviking/master.key文件时，会报错.租户操作
创建租户
创建租户会在
data/viking/test-account/目录下创建一个新的目录，用于存储该租户信息，此时该目录下所有文件应该都被加密.加密文件
$ grep "OVE" -r data/viking/test-account/ Binary file data/viking/test-account/_system/users.json matches Binary file data/viking/test-account/resources/.overview.md matches Binary file data/viking/test-account/resources/.abstract.md matches Binary file data/viking/test-account/user/.overview.md matches Binary file data/viking/test-account/user/admin/.overview.md matches Binary file data/viking/test-account/user/admin/.abstract.md matches Binary file data/viking/test-account/user/admin/memories/preferences/.overview.md matches Binary file data/viking/test-account/user/admin/memories/preferences/.abstract.md matches Binary file data/viking/test-account/user/admin/memories/.overview.md matches Binary file data/viking/test-account/user/admin/memories/.abstract.md matches Binary file data/viking/test-account/user/admin/memories/events/.overview.md matches Binary file data/viking/test-account/user/admin/memories/events/.abstract.md matches Binary file data/viking/test-account/user/admin/memories/entities/.overview.md matches Binary file data/viking/test-account/user/admin/memories/entities/.abstract.md matches Binary file data/viking/test-account/user/.abstract.md matches Binary file data/viking/test-account/agent/.overview.md matches Binary file data/viking/test-account/agent/.abstract.md matches Binary file data/viking/test-account/session/.overview.md matches Binary file data/viking/test-account/session/.abstract.md matches用户操作
注册用户
注册用户会修改
data/viking/test-account/_system/users.json, append 新的用户信息 , 同时在data/viking/test-account/user/目录下创建一个新的目录，用于存储该用户信息，此时该目录下所有文件应该都被加密.加密文件
资源类操作
添加资源
添加资源会在
data/viking/test-account/resources/目录下创建新的文件，用于存储该资源信息，此时该文件应该被加密.$ ov add-resource README.md status success errors [] source_path README.md meta {} root_uri viking://resources/README temp_uri viking://temp/03201957_8ca3dd/README加密文件
$ grep "OVE" -r data/viking/test-account/resources/ Binary file data/viking/test-account/resources/.overview.md matches Binary file data/viking/test-account/resources/README/Advanced_Reading_2more.md matches Binary file data/viking/test-account/resources/README/.overview.md matches Binary file data/viking/test-account/resources/README/Overview.md matches Binary file data/viking/test-account/resources/README/Core_Concepts/.overview.md matches Binary file data/viking/test-account/resources/README/Core_Concepts/.abstract.md matches Binary file data/viking/test-account/resources/README/Core_Concepts/Core_Concepts_4more.md matches Binary file data/viking/test-account/resources/README/Core_Concepts/4_Visualized_Retrieval_Tra_2more.md matches Binary file data/viking/test-account/resources/README/README.md matches Binary file data/viking/test-account/resources/README/.abstract.md matches Binary file data/viking/test-account/resources/README/Quick_Start/.overview.md matches Binary file data/viking/test-account/resources/README/Quick_Start/2_Model_Preparation/.overview.md matches Binary file data/viking/test-account/resources/README/Quick_Start/2_Model_Preparation/2_Model_Preparation_2more.md matches Binary file data/viking/test-account/resources/README/Quick_Start/2_Model_Preparation/.abstract.md matches Binary file data/viking/test-account/resources/README/Quick_Start/2_Model_Preparation/Provider-Specific_Notes.md matches Binary file data/viking/test-account/resources/README/Quick_Start/3_Environment_Configuration/.overview.md matches Binary file data/viking/test-account/resources/README/Quick_Start/3_Environment_Configuration/CLIClient_Configuration_Examples.md matches Binary file data/viking/test-account/resources/README/Quick_Start/3_Environment_Configuration/Server_Configuration_Templ_3more.md matches Binary file data/viking/test-account/resources/README/Quick_Start/3_Environment_Configuration/.abstract.md matches Binary file data/viking/test-account/resources/README/Quick_Start/Prerequisites_2more.md matches Binary file data/viking/test-account/resources/README/Quick_Start/4_Run_Your_First_Example_2more.md matches Binary file data/viking/test-account/resources/README/Quick_Start/.abstract.md matches Binary file data/viking/test-account/resources/README/Server_Deployment_Details_2more.md matches Binary file data/viking/test-account/resources/.abstract.md matches查看资源
查看资源会返回该资源的明文内容.
$ ov abstract viking://resources/README This directory contains documentation and resources for OpenViking, an AI Agent technology project focused on context management and retrieval optimization. It includes community engagement guides, technical overviews, deployment details, and foundation...skills 操作
添加skill
添加资源会在会在
data/viking/test-account/agent/skills/目录下创建新的文件，用于存储该资源信息，此时该文件应该被加密.$ ov add-skill ~/.trae-cn/skills/skill-creator/ status success uri viking://agent/skills/skill-creator name skill-creator auxiliary_files 17加密文件
$ grep "OVE" -r data/viking/test-account/agent/skills/ Binary file data/viking/test-account/agent/skills/skill-creator/eval-viewer/generate_review.py matches Binary file data/viking/test-account/agent/skills/skill-creator/eval-viewer/viewer.html matches Binary file data/viking/test-account/agent/skills/skill-creator/.overview.md matches Binary file data/viking/test-account/agent/skills/skill-creator/references/schemas.md matches Binary file data/viking/test-account/agent/skills/skill-creator/agents/grader.md matches Binary file data/viking/test-account/agent/skills/skill-creator/agents/comparator.md matches Binary file data/viking/test-account/agent/skills/skill-creator/agents/analyzer.md matches Binary file data/viking/test-account/agent/skills/skill-creator/.abstract.md matches Binary file data/viking/test-account/agent/skills/skill-creator/scripts/run_eval.py matches Binary file data/viking/test-account/agent/skills/skill-creator/scripts/package_skill.py matches Binary file data/viking/test-account/agent/skills/skill-creator/scripts/quick_validate.py matches Binary file data/viking/test-account/agent/skills/skill-creator/scripts/improve_description.py matches Binary file data/viking/test-account/agent/skills/skill-creator/scripts/aggregate_benchmark.py matches Binary file data/viking/test-account/agent/skills/skill-creator/scripts/__init__.py matches Binary file data/viking/test-account/agent/skills/skill-creator/scripts/run_loop.py matches Binary file data/viking/test-account/agent/skills/skill-creator/scripts/generate_report.py matches Binary file data/viking/test-account/agent/skills/skill-creator/scripts/utils.py matches Binary file data/viking/test-account/agent/skills/skill-creator/SKILL.md matches Binary file data/viking/test-account/agent/skills/skill-creator/LICENSE.txt matches Binary file data/viking/test-account/agent/skills/skill-creator/assets/eval_review.html matches查看skill
同 resource 查看资源
memory 操作
add-memory 会在
data/viking/test-account/user/test-user/memories目录下创建新的文件，用于存储该资源信息，此时该文件应该被加密.添加memory
$ ov add-memory "我爱吃西瓜" memories_extracted 1加密文件
$ grep "OVE" -r data/viking/test-account/user/test-user/memories/ Binary file data/viking/test-account/user/test-user/memories/preferences/.overview.md matches Binary file data/viking/test-account/user/test-user/memories/preferences/mem_e0f30389-1fe6-49e0-9b73-b686a5c42050.md matches Binary file data/viking/test-account/user/test-user/memories/preferences/.abstract.md matches Binary file data/viking/test-account/user/test-user/memories/.overview.md matches Binary file data/viking/test-account/user/test-user/memories/.abstract.md matches Binary file data/viking/test-account/user/test-user/memories/events/.overview.md matches Binary file data/viking/test-account/user/test-user/memories/events/.abstract.md matches Binary file data/viking/test-account/user/test-user/memories/entities/.overview.md matches Binary file data/viking/test-account/user/test-user/memories/entities/.abstract.md matchessession 操作
session 会在
data/viking/test-account/session/目录下创建新的文件，用于存储该会话信息，此时该文件应该被加密.添加session
$ ov session new session_id f413c71b-4757-47fa-b49e-0e4605846ac4 user {"account_id":"test-account","user_id":"test-user","agent_id":"default"}添加 message
加密文件
$ $ grep "OVE" -r data/viking/test-account/session/ Binary file data/viking/test-account/session/.overview.md matches Binary file data/viking/test-account/session/test-user/907e60e0-f6b2-4f25-aea5-6dc3666fb58b/messages.jsonl matches Binary file data/viking/test-account/session/test-user/907e60e0-f6b2-4f25-aea5-6dc3666fb58b/.overview.md matches Binary file data/viking/test-account/session/test-user/907e60e0-f6b2-4f25-aea5-6dc3666fb58b/.abstract.md matches Binary file data/viking/test-account/session/test-user/907e60e0-f6b2-4f25-aea5-6dc3666fb58b/history/archive_001/messages.jsonl matches Binary file data/viking/test-account/session/test-user/907e60e0-f6b2-4f25-aea5-6dc3666fb58b/history/archive_001/.overview.md matches Binary file data/viking/test-account/session/test-user/907e60e0-f6b2-4f25-aea5-6dc3666fb58b/history/archive_001/.abstract.md matches Binary file data/viking/test-account/session/test-user/f413c71b-4757-47fa-b49e-0e4605846ac4/messages.jsonl matches Binary file data/viking/test-account/session/.abstract.md matchesrelation 操作
link resourceA resourceB会在
data/viking/test-account/resources/A目录下创建新的文件，用于存储该会话信息，此时该文件应该被加密.添加relation
加密文件
$ grep "OVE" data/viking/test-account/resources/README/.relations.json Binary file data/viking/test-account/resources/README/.relations.json matches查看relation
通过 relations 查看该资源的所有关联资源.