[ERROR] Errors after updating to 0.17 (latest)

[byronz3d]

Hi,

I upgraded to latest version 0.17
When starting Zep I get the following errors:

zep    | time="2023-11-01T15:49:52Z" level=info msg="Starting Zep server version 0.17.0-c8702ca (2023-10-31T14:11:05+0000)"
zep    | time="2023-11-01T15:49:52Z" level=info msg="Log level set to: info"
zep    | time="2023-11-01T15:49:53Z" level=info msg="vector extension version is >= 0.5.0. hnsw indexing available"
zep    | time="2023-11-01T15:49:53Z" level=fatal msg="unable to create memoryStore storage error: failed to run OnInit (original error: storage error: failed to ensure postgres schema setup (original error: error checking summary embedding dimensions: error getting embedding column width: error getting embedding column width: sql: no rows in result set))"
zep exited with code 1

And it will continue showing that.

I downgraded back to 0.16 and it works again

Is it something I can fix on my end ?

[danielchalef]

Thanks for reporting. I've seen another mention of this issue, but am having trouble replicating it with an upgrade from 0.16.0 and 0.16.1.

Which Zep version were you upgrading from?
Are you using Zep's Postgres database container or a managed database service? If the latter, what version of Postgres are you using?

[byronz3d]

The last time I updated Zep was 3 weeks ago when you fixed the other issue I had. I am not sure what version it was.
I am using Zep's Postgres database that comes with the Zep docker container.

[byronz3d]

with 0.16 I suddenly get

zep_python.exceptions.APIError: API error: {'status_code': 500, 'message': 'failed to Create collection: failed to insert collection: ERROR: null value in column "index_type" of relation "document_collection" violates not-null constraint (SQLSTATE=23502)\n'}

So I downgraded to 0.15.2 and error is gone.

So maybe I have something in my 0.15.2 database that doesn't work with newer version ?

[danielchalef]

Would you be able to capture all debug logs when upgrading to 0.17.0 and attach them to this issue? Steps below:

1. Add the following line to your .env file:
   ZEP_LOG_LEVEL=debug
2. Run docker compose up, but capture logs to a log file:
   docker compose up |& tee logs.txt
3. Open logs.txt in a text editor and delete anything you view and sensitive.

Thanks!

[byronz3d]

logs.txt
Attached

[danielchalef]

@byronz3d thanks for sharing the log files. I've built a bug fix release of Zep that creates the missing column if it doesn't exist. I'm still not entirely certain why this occurred. Would you like to try again with Zep v0.17.2?

[byronz3d]

Upgraded to 0.17.2
New log file attached
logs.txt

[danielchalef]

Aha! Very helpful. Thanks. Please would you follow the upgrade instructions at the bottom of this announcement i.e. add the config keys / environment variables? https://blog.getzep.com/zep-v0-17/

Additionally, run a git pull in the Zep repo directory.

What's happening is that your config file is outdated and you're missing the embedding dimensions config keys, which are being read as 0. We'll make this less to prone to failure.

�[36mzep    |�[0m time="2023-11-01T22:09:11Z" level=warning msg="summary embedding dimensions are 1536, expected 0.\n migrating summary embedding column width to 0. this may result in loss of existing embedding vectors"

[byronz3d]

@danielchalef I updated the Zep repo as well and now the previous errors are gone.
Now I get:

zep    | time="2023-11-01T22:59:23Z" level=fatal msg="store.type must be set"

Even though in config.yaml from the repo I have

store:
  type: "postgres"

I tried adding
-ZEP_STORE_TYPE
to environment and .env files, but no help

Also it looks like it's ignoring my .env file because I have
ZEP_LOG_LEVEL=debug
But when starting Zep:

zep    | time="2023-11-01T22:59:23Z" level=info msg="Log level set to: info"

[danielchalef]

Would you please share your docker compose file?

[byronz3d]

docker-compose.txt
Attached

[danielchalef]

I'm able to start Zep using your docker-compose.yaml file and the config.yaml file currently in this repo.

Note that your compose file has duplicate shm values for the db service and one had to be removed in order for docker to successfully start the services.

Please would you recheck your compose file, config, and .env to ensure that there isn't something in them that would be causing the new error?

[byronz3d]

It works now after resetting the config.yaml file to default
It seems like this is causing an issue
secret: "1o~~UgE+&xa%/#cB,rvj#5_Soo1NlYJmH\2ugmrMZaj,9YHWHW#+h=|,mQpjbDA'Nm7IXV&tI-3ysk~5EC-v4,=rw@wAHo0lrbB#zNP6Pj08WirUV2w/vhB&jB:qoHiFG?qQ/*.WE-N^'|.$S:Pf"
So I just set it to a less complicated string and the previous error is gone.

Thanks for all the support brother!

[danielchalef]

I recommend not storing the secret in your config file, but rather in a secrets manager that injects the secret as an environment variable wherever you are running Zep. Config files can be accidentally added to source control and/or be accessed by bad actors.