…blocks during decommissioning

CR feedback "Nit: I think the comment is not needed as your code is self-explanatory here:"

Co-authored-by: Attila Zsolt Piros <2017933+attilapiros@users.noreply.github.com>

…Resolver.scala


If we have a failure during block migration, log the exception.

Co-authored-by: Attila Zsolt Piros <2017933+attilapiros@users.noreply.github.com>

…ment

…imeout here

Co-authored-by: Attila Zsolt Piros <2017933+attilapiros@users.noreply.github.com>

Co-authored-by: Attila Zsolt Piros <2017933+attilapiros@users.noreply.github.com>

Co-authored-by: Attila Zsolt Piros <2017933+attilapiros@users.noreply.github.com>

Co-authored-by: Attila Zsolt Piros <2017933+attilapiros@users.noreply.github.com>

Co-authored-by: Attila Zsolt Piros <2017933+attilapiros@users.noreply.github.com>

Co-authored-by: Attila Zsolt Piros <2017933+attilapiros@users.noreply.github.com>

Co-authored-by: Attila Zsolt Piros <2017933+attilapiros@users.noreply.github.com>

… well (help avoid cascading block migration)

…hreads during testing.

…he and log the scheduler when asked to decom and we can't

…RemoteBlockSizeFetchToMem test that we don't need anymore, add back in submitting the thread I accidently took out in applying some CR feedback (a little fast on the ctrl-k)

…ore the execution service being used to run the threads so we can stop them explicitly. Still uses a graceful stop for other executors which become no longer healthy targets instead of a thread kill.

…mmissionManager

…rence is clear from the class name (integration-style/unit-style)

…ager to BlockManagerDecommissioner

…r on what were passing around than just rabndom tuples

…resolver, fix string interopolation, remove un-needed retry logic in decommissioning, etc.

…rs to limit the number of concurrent migrations. We still limit to one runnable per peer executor so we don't overwhelm any particular target even at large values of the thread pool.

…ation

…t number of threads to match the other similar config, add some comments, make isShuffle treat internalShuffle's the same simplifying the code some, seperate the shuffle block refresh and rdd block threads, rever un-needed change to SparkContext, and other misc cleanups

…re-being-shutdown-cleaned-up

…re-being-shutdown-cleaned-up

…re-being-shutdown-cleaned-up

…all the executors to come up first only slows the entire suite down by ~4s and should remove that flake

…els, add a Since annotaiton.

… need an explicit test for it here and the block manager is stubbed out in IndexShuffleBlockResolverSuite

…re-being-shutdown-cleaned-up