Adds two-phase deletion for physical shards via the shard manager service. deleteShard will mark a row as deleted, rather than dropping it. there are two additional rpcs, getDeletedShards and purgeShard.
getDeletedShards will return all shards marked as deleted, and purgeShard will delete the row, and call purge on the appropriate shard factory.
changes not involving sqlshard
sql shard changes
smarter deleteShard method
make NameServer not extend shards.Shard
tests for deleting and purging logical and physical shards
factor out stubbed materialize and purge into AbstractShardFactory
I forgot to add, this involves both a name server schema change, as well as new stuff in the shard manager service thrift interface.
uncomment test assertion
Merge branch 'master' into shard_purging
in SqlShard#deleteShard: if a factory is an AbstractShardFactory, you call purge immediately. is that because any descendant of AbstractShardFactory should be auto-purged? if so, can you give it a name like AutoPurgeShardFactory or something?
slightly worried that shardsForHostname might be slower if you fetch tens of thousands of shards and THEN filter by hostname on the client.
cool to get rid of the Busy converter. :)
reshuffle things, re-expand shardsForHostname to be efficient
The logic in sql shard makes this less clear, but materialize and purge are stubbed out in AbstractShardFactory to do nothing. I can fix it so we do not rely on purge to be a no-op in those cases and instead never call it for an AbstractShardFactory. The same logic should be applied to materialize in that case.
Fixed shardsForHostname to filter in the db.
more clear purge logic in nameserver shard
add mock calls
fix abstract shard deletes
purge and delete logic cleared up. one of these days, we should clean up the nameserver sql shard for real, but not this time, I guess.
i think i'm just being confused by the word "abstract", which normally means a base class with some default implementation, but in this case i think what you really mean is "class that doesn't want the two-phase delete feature". if you renamed that class, it would be clearer.
True, what I really mean is: "These things are ephemeral and do not have a backing data store." As opposed to the desirability of any specific behavior. Any good ideas for naming that?
hey, I like virtual. that makes sense.
rename AbstractShardFactory to VirtualShardFactory, as that is more c…
…lear as to purpose