kvrocks2redis run failed

[iushas]

Search before asking

• I had searched in the issues and found no similar issues.

Version

unstable

Minimal reproduce step

run kvrocks2redis -c kvrocks2redis.conf

What did you expect to see?

success

What did you see instead?

coredump

• gdb kvrocks2redis ./core

Core was generated by `kvrocks2redis -c kvrocks2redis.conf'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000557a43551a96 in engine::Storage::LatestSeqNumber (this=<optimized out>) at incubator-kvrocks/src/storage/storage.cc:529
529 incubator-kvrocks/src/storage/storage.cc: No such file or directory.

• there is no error in logfile

Log file created at: 2024/01/05 15:03:37
Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg
I20240105 15:03:37.463881 3476657 main.cc:109] kvrocks2redis unstable (commit 230d9726)
I20240105 15:03:37.832939 3476657 storage.cc:343] [storage] Success to load the data from disk: 366 ms
I20240105 15:03:37.833065 3476657 sync.cc:61] [kvrocks2redis] Start sync the data from kvrocks to redis
~

Anything Else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[maochongxin]

Hi @iushas, can you provide the reproduction steps or more stack trace information?

[iushas]

• 1、compile kvrocks2redis only
  

maybe this step has something wrong? some codes do not been compiled in program?
but version v2.7.0 is normal.

• 2、run kvrocks2redis -c kvrocks2redis.conf

log-level INFO
daemonize no
data-dir /home/redis/data
output-dir /home/redis

cluster-enabled no

namespace.__namespace ip 6379 1234567

[iushas]

In addition, the old version file contains the configuration information of the source instance. Is there no need to configure it now?

[maochongxin]

In addition, the old version file contains the configuration information of the source instance. Is there no need to configure it now?

Yes, in the new version, kvrocks2redis uses the SecondaryMode to open the RocksDB instance for synchronization instead of using network synchronization for incremental data. Therefore, in the configuration, the data-dir is set to the working directory of kvrocks

[iushas]

In addition, the old version file contains the configuration information of the source instance. Is there no need to configure it now?

Yes, in the new version, kvrocks2redis uses the SecondaryMode to open the RocksDB instance for synchronization instead of using network synchronization for incremental data. Therefore, in the configuration, the data-dir is set to the working directory of kvrocks

OK, I'm sure the working directory of kvrocks is configured correctly

[iushas]

Is this bug resolved? @maochongxin @git-hulk

[maochongxin]

Hi @iushas , I have tested and it runs smoothly without encountering your issue. Here are the steps I followed for my testing:

commit: e95e6f7

# build
./x.py build

# start kvrocks
./kvrocks -c kvrocks.conf

# start redis
redis-server

# start kvrocks2redis
./kvrocks2redis -c kvrocks2redis.conf

# test
python3 populate-kvrocks.py --flushdb=true --password=''
python3 check_consistency.py --src_password=''

kvrocks2redis.conf , kvrocks working directory /tmp/kvrocks

################################ GENERAL #####################################

# The value should be INFO, WARNING, ERROR, FATAL.
#
# Default: INFO
log-level INFO

# By default kvrocks2redis does not run as a daemon. Use 'yes' if you need it.
# Note that kvrocks2redis will write a pid file in /var/run/kvrocks2redis.pid when daemonized.
#
# Default: no
daemonize no

# The kvrocks working directory.
# Note that you must specify a directory here, not a file name.
#
# Default: ./data
data-dir /tmp/kvrocks

# Intermediate files are output to this directory when the kvrocks2redis program runs.
#
# Default: ./
output-dir ./


# Enable cluster mode.
#
# Default: no
cluster-enabled no

################################ NAMESPACE AND Sync Target Redis #####################################
# Synchronize the specified namespace data to the specified Redis DB.
# Warning: It will flush the target redis DB data.
#
# namespace.{namespace} <redis_ip> <redis_port> [<redis_auth> <redis_db_number>]
# Default redis_db_number is 0
namespace.__namespace 127.0.0.1 6379

[iushas]

Hi @iushas , I have tested and it runs smoothly without encountering your issue. Here are the steps I followed for my testing:

commit: e95e6f7

# build
./x.py build

# start kvrocks
./kvrocks -c kvrocks.conf

# start redis
redis-server

# start kvrocks2redis
./kvrocks2redis -c kvrocks2redis.conf

# test
python3 populate-kvrocks.py --flushdb=true --password=''
python3 check_consistency.py --src_password=''

kvrocks2redis.conf , kvrocks working directory /tmp/kvrocks

################################ GENERAL #####################################

# The value should be INFO, WARNING, ERROR, FATAL.
#
# Default: INFO
log-level INFO

# By default kvrocks2redis does not run as a daemon. Use 'yes' if you need it.
# Note that kvrocks2redis will write a pid file in /var/run/kvrocks2redis.pid when daemonized.
#
# Default: no
daemonize no

# The kvrocks working directory.
# Note that you must specify a directory here, not a file name.
#
# Default: ./data
data-dir /tmp/kvrocks

# Intermediate files are output to this directory when the kvrocks2redis program runs.
#
# Default: ./
output-dir ./


# Enable cluster mode.
#
# Default: no
cluster-enabled no

################################ NAMESPACE AND Sync Target Redis #####################################
# Synchronize the specified namespace data to the specified Redis DB.
# Warning: It will flush the target redis DB data.
#
# namespace.{namespace} <redis_ip> <redis_port> [<redis_auth> <redis_db_number>]
# Default redis_db_number is 0
namespace.__namespace 127.0.0.1 6379

Still having problems, can you send me your binary file to try?

[PragmaTwice]

@iushas Could you attach your coredump file and kvrocks2redis binary?

And it would be very helpful if you put the crash stacktrace here. (type bt in gdb)