[MINOR][CORE] Improved statistical shuffle write time

[heary-cao]

What changes were proposed in this pull request?

Creating the file to write to and creating a disk writer both involve interacting with the disk, and can take a long time when we open or close many files, so should be included in the shuffle write time.

so we call mergeSpillsWithTransferTo, only contains the write file the time, but did not included in the shuffle write time when open and close many merges spill files .

How was this patch tested?

existed test cases.

[jiangxb1987]

Personally I don't think we should take the file open/close time into account, cc @cloud-fan .

[cloud-fan]

I don't have a strong preference, cc @srowen

[jerryshao]

Whether shuffle write time should include the file open/close time is debatable, also we don't know whether the actual open action is lazy or not (depends on OS). But one downside of this change is that it makes this metric incomparable to the old one, because we changed the semantics now.

[jiangxb1987]

Seems it's better to keep the previous behavior.

[heary-cao]

@cloud-fan @jerryshao @jiangxb1987 thank you for review it.
Whether shuffle write time should include the file open/close time is debatable I agree with you @jerryshao . Because review spark code, see the realization of this controversy, But this PR hope to unify.

[cloud-fan]

This can be a debate when we introduced this metric, now I think it's too late to change.

[HyukjinKwon]

Shall we leave this closed for now @heary-cao?

[AmplabJenkins]

Can one of the admins verify this patch?