Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Guarantee issues coming randomly #6623

Open
fallais opened this issue Feb 21, 2018 · 0 comments
Open

Guarantee issues coming randomly #6623

fallais opened this issue Feb 21, 2018 · 0 comments

Comments

@fallais
Copy link

fallais commented Feb 21, 2018

Hello,

We are running RethinkDB with Docker, everything worked well since a few days and now the container is crashing randomly because of "guarantee" issues.
Once it is crashed, the container never restart again, like if the database is corrupted.
We need to restore a previous snapshot.
And it is crashing again a few days later.

Here are the logs.

Loading data from directory /data/rethinkdb_data
Listening for intracluster connections on port 29015
Listening for client driver connections on port 28015
Listening for administrative HTTP connections on port 8080
Listening on cluster addresses: 127.0.0.1, 172.18.0.2
Listening on driver addresses: 127.0.0.1, 172.18.0.2
Listening on http addresses: 127.0.0.1, 172.18.0.2
Server ready, "8800ff073542_pln" eec97d08-c89f-4c24-a047-bc69f16e91f7
Version: rethinkdb 2.3.6~0jessie (GCC 4.9.2)
error: Error in src/serializer/log/data_block_manager.cc at line 1202:
error: Guarantee failed: [gc_state->current_entry == nullptr] 0x7fb55bf42780: 2080768 garbage bytes left on the extent, 16384 index-referenced bytes, 0 token-referenced bytes, at offset 16777216.  block dump:
       [18546688..+4096)  I
       [18550784..+4096)  I
       [18554880..+4096)  I
       [18558976..+4096)  I
error: Backtrace:
error: Tue Feb 20 09:26:08 2018
 
       1 [0x7d6e20]: backtrace_t::backtrace_t() at 0x7d6e20 (rethinkdb)
       2 [0x7d7199]: format_backtrace(bool) at 0x7d7199 (rethinkdb)
       3 [0x72b053]: report_fatal_error(char const*, int, char const*, ...) at 0x72b053 (rethinkdb)
       4 [0x77d3f6]: data_block_manager_t::gc_one_extent(data_block_manager_t::gc_state_t*) at 0x77d3f6 (rethinkdb)
       5 [0x77e20b]: data_block_manager_t::run_gc(data_block_manager_t::gc_state_t*) at 0x77e20b (rethinkdb)
       6 [0x815857]: coro_t::run() at 0x815857 (rethinkdb)
error: Exiting.

Here is Docker

root@xxxxxxxx# docker version
Client:
 Version:       17.12.0-ce
 API version:   1.35
 Go version:    go1.9.2
 Git commit:    c97c6d6
 Built: Wed Dec 27 20:11:19 2017
 OS/Arch:       linux/amd64

Server:
 Engine:
  Version:      17.12.0-ce
  API version:  1.35 (minimum version 1.12)
  Go version:   go1.9.2
  Git commit:   c97c6d6
  Built:        Wed Dec 27 20:09:53 2017
  OS/Arch:      linux/amd64
  Experimental: false

Here is the OS.

root@xxxxxxxxx# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial

Thanks a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants