Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shard metadata never be expired #767

Closed
wants to merge 1 commit into from
Closed

Shard metadata never be expired #767

wants to merge 1 commit into from

Conversation

jvshahid
Copy link
Contributor

@jvshahid jvshahid commented Aug 1, 2014

I tried automatic shard expiration with the following database configuration:

[
{
  "database": "test",
  "spaces": [
    {
      "name": "everything",
      "retentionPolicy": "10m",
      "shardDuration": "1m",
      "regex": "/.*/",
      "replicationFactor": 2,
      "split": 1
    }
  ]
}
]

Then, old shards were successfully dropped:

[07/18/14 11:43:36] [INFO] DATASTORE: dropping shard /home/shugo/work/influxdb/cluster_test/1/db/shard_db_v2/00003

However, "GET /cluster/shards" returns dropped shards.

I guess shard metadata should be removed by updateOrRemoveShard() in all nodes, or shards should be dropped through Raft.

@pauldix
Copy link
Member

pauldix commented Aug 1, 2014

Dropping of a shard from a server should go through raft. That way all the other servers in the cluster will remove the reference of that shard being kept on a given server.

So drop shard 1 from server 1, or whatever. If the drop is for the local server id, then actually delete it from disk. If the drop is the last server to have a copy of the shard, remove the shard entirely from the cluster config.

shugo added a commit to shugo/influxdb that referenced this pull request Aug 6, 2014
@shugo
Copy link
Contributor Author

shugo commented Aug 6, 2014

@pauldix I've created PR #803 by which expired shards are dropped through raft. Could you check it?

@jvshahid jvshahid closed this in 7a297b3 Aug 7, 2014
@jvshahid jvshahid deleted the fix-767 branch August 12, 2014 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants