[Nikhil ad-hoc] [2] (4) Trying to access db_version when starting up kelp with db before the table even exists?

[nikhilsaraf]

Reported by someone in the community. Need to look into this.

// see a sample bug report here: #1

Describe the bug

// A succinct description of the buggy behavior.
// This section includes what the bot is doing and why you think it is incorrect.

The bot ...

Expected behavior

// A succinct description of what you expected to happen.

I think the correct behavior in this situation is for the bot to ...

Frequency

// How often does this happen?
// Unpredictable = it's hard to predict when it will happen
// Sometimes = it happens every now and then but there's no pattern to it
// Always = it always happens consistently at the same point

The frequency is: ...

Steps To Reproduce

// A concise description of the steps to reproduce the buggy behavior along with screenshots wherever applicable:

Here are the steps to reproduce the issue (see attachments in section below):

1. Set up your configuration file with these parameters: ...
2. Start the bot using this command: ...
   ...

Possible Solution

// Optional, suggest a fix.

This bug can be fixed by ...

Your Environment

// Run kelp version to get the version and build information and paste it here
// cli version: v1.0.0-rc1
// gui version: v0.0.1
// git hash: 1bcc4e9
// build date: 20180813T013016Z
// GOOS: linux
// GOARCH: amd64

...

Context

// A succinct description of how has this bug has affected you or prevented you from accomplishing what you wanted.

I am unable to ... because of this bug.

Attachments

// Attach any relevant configuration files, logs, tx hashes, etc. here.

[nikhilsaraf]

This is a bug report from the community. They are running kelp with the postgres feature enabled from the config file. They are unable to start kelp because it tries to access the db_version table although the db_version table does not exist in their db.

IIRC all versions of kelp build the db_version as part of the schema from version 1 of the db onwards. The task in this issue is to repro the above and confirm whether this is a bug or not and if it's a bug to find the solution and fix it.

[theclabs]

Describe the bug
at initial kelp fails to create DB when process start

Expected behavior
when starting for the first time kelp must create the db and initialize it

Frequency
The frequency is: Always = it always happens consistently at the same point

Steps To Reproduce
// A concise description of the steps to reproduce the buggy behavior along with screenshots wherever applicable:
Here are the steps to reproduce the issue (see attachments in section below):

On Ubuntu 18
download
$ wget https://github.com/stellar/kelp/releases/download/v1.9.0/kelp-v1.9.0-linux-amd64.tar
unzip
$tar xvf kelp-v1.9.0-linux-amd64.tar
modify sample_trader with the DB config
[POSTGRES_DB]
HOST = "192.168.1.100"
PORT = 5432
DB_NAME = "localnewkelp"
USER = "XXX"
PASSWORD = "XXX"
SSL_ENABLE = false
( the database works and receives connections from the machine where I run kelp)
dont forget to set FILL_TRACKER_SLEEP_MILLIS when use a DB
Start the bot using this command: ./kelp trade -c sample_trader.cfg -s buysell -f sample_buysell.cfg

and watch crash 😢
in first run the error is
problem encountered while initializing the db: could not run upgrade scripts: could not fetch current db version: could not execute sql select query (SELECT version FROM db_version ORDER BY version desc LIMIT 1): pq: no existe la relación «db_version»

and in a second run try the error is
problem encountered while initializing the db: error when creating database from config ({Host:localhost Port:5432 DbName:localnewkelp User:postgres Password:XXXXX SSLEnable:false}), created=false: could not create database 'localnewkelp': pq: la base de datos «localnewkelp» ya existe

Possible Solution
// Optional, suggest a fix.

I think this error can be solved if the correct order of the creation of the database is verified, since I guess, kelp try to access the DB version before inserting these records.
currently I don't have the code since I don't specialize in golang and I have encountered problems setting up the environment necessary to do a deeper debug.

Environment
cli version: master:v1.9.0
gui version: v1.0.0-rc1
git branch: master
git hash: ab3333e
build date: 20200507T143536Z
env: release
GOOS: linux
GOARCH: amd64
.

Context
I cannot operate the bot because I cannot take advantage of the functionality of having a record of the transactions made by it.
Attachments

It should be clarified that I use the demo configuration files for this test, but the same thing happens to me if I configure the files for other strategies, I have tried with buysell and mirror.

im using postgres on windows
PostgreSQL 11.5, compiled by Visual C++ build 1914, 64-bit

[theclabs]

@debnil Hello again, for anything you need to know about this error or do a related test, please let me know!
It would be very helpful for me and others to be able to contribute to the solution of this bug

[debnil]

@theclabs will do! On my docket for the next couple days, so expect a message soon if I need anything else :)

[debnil]

@theclabs: Unfortunately, I was unable to reproduce this from the tarfile. (Note that I used darwin/OS X, not Linux, on my own setup.)

Have you confirmed the correctness of your Postgres username and password, separately and before using it on Kelp? I did that and had no issues creating a new database called localnewkelp:

2020/10/15 16:46:55 created database 'localnewkelp'
2020/10/15 16:46:55 creating db schema and running upgrade scripts ...
2020/10/15 16:46:55    executed sql statement at index 0 for db version 1
2020/10/15 16:46:55    successfully ran 1 upgrade commands and upgraded to version 1 of the database in 3 milliseconds
2020/10/15 16:46:55    executed sql statement at index 0 for db version 2
2020/10/15 16:46:55    executed sql statement at index 1 for db version 2
2020/10/15 16:46:55    executed sql statement at index 2 for db version 2
2020/10/15 16:46:55    successfully ran 3 upgrade commands and upgraded to version 2 of the database in 6 milliseconds
2020/10/15 16:46:55    executed sql statement at index 0 for db version 3
2020/10/15 16:46:55    executed sql statement at index 1 for db version 3
2020/10/15 16:46:55    successfully ran 2 upgrade commands and upgraded to version 3 of the database in 2 milliseconds
2020/10/15 16:46:55 ... finished creating db schema and running upgrade scripts
2020/10/15 16:46:55 made db instance with config: host=localhost port=5432 sslmode=disable user=postgres dbname=localnewkelp

Past that, I'm not too certain why this bug would surface in Linux but not OS X.

Tagging in @nikhilsaraf for any thoughts on other ways to debug.

[nikhilsaraf]

Looking into this. I got a SQL error when running under linux with postgres 9.3.24, maybe it's with the SQL version.

@theclabs can you try running with SQL 12.1+?

our circle ci instance runs on Linux and uses 12.1

[theclabs]

@nikhilsaraf Hello, I just verified, the current version I had is PostgreSQL 11.5 in windows.
I installed the PostgreSQL 13.0 version in ubuntu and it works!!
maybe it should be clarified in the readme that to use PSQL with only versions 12.1 +

[nikhilsaraf]

@theclabs added to README, thanks for filing the issue and let me know if you run into any more issues with Kelp :)