Skip to content

Two-phase deletion for physical shards #18

Open
wants to merge 17 commits into from

2 participants

@freels
freels commented Sep 22, 2010

Adds two-phase deletion for physical shards via the shard manager service. deleteShard will mark a row as deleted, rather than dropping it. there are two additional rpcs, getDeletedShards and purgeShard.

getDeletedShards will return all shards marked as deleted, and purgeShard will delete the row, and call purge on the appropriate shard factory.

@freels
freels commented Sep 22, 2010

I forgot to add, this involves both a name server schema change, as well as new stuff in the shard manager service thrift interface.

@robey
robey commented Oct 4, 2010

in SqlShard#deleteShard: if a factory is an AbstractShardFactory, you call purge immediately. is that because any descendant of AbstractShardFactory should be auto-purged? if so, can you give it a name like AutoPurgeShardFactory or something?

slightly worried that shardsForHostname might be slower if you fetch tens of thousands of shards and THEN filter by hostname on the client.

cool to get rid of the Busy converter. :)

@freels
freels commented Oct 5, 2010

The logic in sql shard makes this less clear, but materialize and purge are stubbed out in AbstractShardFactory to do nothing. I can fix it so we do not rely on purge to be a no-op in those cases and instead never call it for an AbstractShardFactory. The same logic should be applied to materialize in that case.

Fixed shardsForHostname to filter in the db.

@freels
freels commented Oct 5, 2010

purge and delete logic cleared up. one of these days, we should clean up the nameserver sql shard for real, but not this time, I guess.

@robey
robey commented Oct 5, 2010

i think i'm just being confused by the word "abstract", which normally means a base class with some default implementation, but in this case i think what you really mean is "class that doesn't want the two-phase delete feature". if you renamed that class, it would be clearer.

@freels
freels commented Oct 5, 2010

True, what I really mean is: "These things are ephemeral and do not have a backing data store." As opposed to the desirability of any specific behavior. Any good ideas for naming that?

@robey
robey commented Oct 5, 2010

hmm....

SymbolicShardFactory? VirtualShardFactory?

@freels
freels commented Oct 5, 2010

hey, I like virtual. that makes sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.