Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for arbitrary number of shard spaces with set retention policies #571

Closed
pauldix opened this issue May 22, 2014 · 3 comments
Closed

Add support for arbitrary number of shard spaces with set retention policies #571

pauldix opened this issue May 22, 2014 · 3 comments
Assignees
@pauldix

Comments

@pauldix
Copy link
Member

pauldix commented May 22, 2014

Currently we have short term and long term shards and data is assigned by what the first character is. We should change this so that assignment is set by database and a regex. Each shard space should have a data retention policy set as a duration. The database should automatically drop shards that are past the retention policy in any given shard space.

See this thread for more info: https://groups.google.com/forum/#!topic/influxdb/4aC0AMPCcFs
@pauldix pauldix self-assigned this May 22, 2014
@pauldix pauldix mentioned this issue May 22, 2014
Closed
@schmurfy
Copy link
Contributor

this looks really nice :)

@schmurfy
Copy link
Contributor

Hi, I just got back from vacations, is there any update on this ?
I checked the changelog for the versions released when I was away but the only thing indirectly related is the compaction fix on deletes which may allow me to finally delete my old data I have to check.

@pauldix
Copy link
Member Author

pauldix commented Jun 16, 2014

I'm working on this and a few other things this week. Hope to have it done
soon

On Mon, Jun 16, 2014 at 6:45 AM, Julien Ammous notifications@github.com
wrote:

Hi, I just got back from vacations, is there any update on this ?
I checked the changelog for the versions released when I was away but the
only thing indirectly related is the compaction fix on deletes which may
allow me to finally delete my old data I have to check.

—
Reply to this email directly or view it on GitHub
#571 (comment).

@pauldix pauldix mentioned this issue Jul 9, 2014
Closed
pauldix added a commit that referenced this issue Jul 10, 2014
@pauldix
Add Shard Spaces and Retention Policies
95b951d 
Fixes #571. Removed the old long term and short term shards. Now if the user doesn't manually create a shard space, a catchall called "default" will be created the first time they write data in.

Added API endpoints to list, create, and delete shard spaces. Made a breaking change to the old get shards API by changing the format since shortTerm and longTerm shards aren't guaranteed to be present.

Shard spaces can be mapped to either a single database or any number of databases. Retention policies are enforced per server by a background routine that periodically checks shards in spaces that have retention policies to see if they have passed their expiration.
pauldix added a commit that referenced this issue Jul 11, 2014
@pauldix @jvshahid
Add Shard Spaces and Retention Policies
155b3be 
Fixes #571. Removed the old long term and short term shards. Now if the user doesn't manually create a shard space, a catchall called "default" will be created the first time they write data in.

Added API endpoints to list, create, and delete shard spaces. Made a breaking change to the old get shards API by changing the format since shortTerm and longTerm shards aren't guaranteed to be present.

Shard spaces can be mapped to either a single database or any number of databases. Retention policies are enforced per server by a background routine that periodically checks shards in spaces that have retention policies to see if they have passed their expiration.

Conflicts:
	src/cluster/cluster_configuration.go
@majst01 majst01 mentioned this issue Jul 15, 2014
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pauldix pauldix

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@schmurfy @pauldix