复现PersonaMEM-V2 #216
Unanswered
YanZiBuGuiCHunShiWan
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
EverMemOS Profile-only ablation for PersonaMem-v2.
Usage:
uv run python -m evaluation.cli --dataset personamemv2 --system evermemos_only_profile --run-name profile-only
name: "evermemos_only_profile"
version: "1.0"
description: "EverMemOS PersonaMem-v2 profile-only ablation"
adapter: "evermemos"
llm:
provider: "${LLM_PROVIDER:openai}"
model: "${LLM_MODEL:gpt-4.1-mini-2025-04-14}"
api_key: "${LLM_API_KEY:xxxxxx}"
base_url: "${LLM_BASE_URL:xxxxx}"
temperature: ${LLM_TEMPERATURE:0.3}
max_tokens: ${LLM_MAX_TOKENS:32768}
search:
mode: "agentic"
dataset_overrides:
personamemv2:
add:
enable_profile_extraction: true
enable_clustering: true
enable_event_log_extraction: false
max_concurrent_requests: ${EVALUATION_ADD_MAX_CONCURRENT:4}
profile_extraction_mode: "scene"
profile_scenario: "assistant"
profile_min_confidence: 0.6
profile_min_memcells: 1
profile_life_max_items: 25
answer:
use_profile_in_answer: true
use_episode_in_answer: false
use_profile_classifier: true
按照这个配置跑Locomo-benchmark 分支的PersonaMEM-V2 复现的效果只有38%,请问是需要修改什么配置么?
Beta Was this translation helpful? Give feedback.
All reactions