Clarification on OmniMemEval's “Viking” evaluation result

[zhangxiaoyan1081]

Hi MemOS / OmniMemEval team,

I’m from the OpenViking team.

We noticed that the OmniMemEval User Memory results include a backend named “Viking”, marked as a cloud deployment, while the evaluation guide refers to the corresponding adapter as “Viking Memory” (--lib viking).

Could you help clarify which exact product or service was evaluated?

• The OpenViking open-source project: https://github.com/volcengine/OpenViking
• Another Volcengine viking memory service
• Or another product with a similar name

We are asking because the reproduced results currently reported for “Viking”—including 69.33 on LoCoMo and 61.07 on LongMemEval—differ materially from the results we have obtained for OpenViking under our current evaluation setup.

We do not want to assume that the results are directly comparable or that there is an issue with the benchmark. The difference may come from the evaluated product, version, adapter implementation, or evaluation configuration.

If the evaluated backend was OpenViking, could you please share or point us to the following details?

• The OpenViking version or commit, deployment type, and approximate evaluation date
• The viking adapter implementation and product-specific configuration
• How benchmark conversations were mapped to OpenViking users and sessions
• Whether session commit and asynchronous memory extraction were completed before retrieval
• Any backend-specific settings beyond OmniMemEval’s shared answer and judge model configuration

This information would help us reproduce the result and determine whether the difference comes from product identity, version, or protocol.

We would be happy to help review the adapter, provide a reproducible OpenViking configuration, or contribute a PR. After the product identity is confirmed, it may also be helpful to:

• Rename the row to the full product name if it does not refer to OpenViking; or
• Add the OpenViking repository link and evaluated version if it does.

Thank you again for open-sourcing OmniMemEval and for making cross-product memory evaluation easier. We look forward to discussing the setup with you.