Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

We shouldn't try to take two snapshots at the same time #943

Closed
jvshahid opened this issue Sep 16, 2014 · 4 comments
Closed

We shouldn't try to take two snapshots at the same time #943

jvshahid opened this issue Sep 16, 2014 · 4 comments
Assignees

Comments

@jvshahid
Copy link
Contributor

As mentioned in this thread, if we tried to take two snapshots at the same time, the second snapshot will error out with Last snapshot is not finished. TakeSnapshot() in raft_server.go should lock to make sure that we don't try to take multiple snapshots at the same time. Note that taking two snapshots in a row doesn't necessarily do extra work, since raft will ignore the command if no commands were committed since the old snapshot.

@consense
Copy link

+1

@dgnorton
Copy link
Contributor

Can be reproduced using https://github.com/dgnorton/influxdb_stress.

  • create a queries.txt file with just one query in it drop series {{randSeries}}
  • run ./influxdb_stress -q queriest.txt

@toddboom
Copy link
Contributor

@dgnorton This might be related to the issue I saw last night. It seems that lots of drop series under heavy read/write load was able to get a cluster into a wonky state.

@dgnorton
Copy link
Contributor

@toddboom investigating now to see if more than one server from raft_server.go is being created.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants