Spark: Extend commit unknown exception handling to SparkPositionDeltaWrite

[singhpk234]

About the change

Extend the handling for #4687 for SparkPositionDeltaWriter used in MOR.

Testing Done

Added an E2E UT as done in earlier PR.

---

cc @RussellSpitzer @aokolnychyi @flyrain

[RussellSpitzer]

@singhpk234 you probably saw how I wrote the test in the other suite, it is not pretty but I think you maybe be able to pull something off like that as well. Although this probably won't be easy unless it is coupled with a Spark version that natively supports the merge operations (3.3 I believe @aokolnychyi has it merged in) since at the moment we would need custom plan rules as well I think ...

I'm getting close to thinking we need byteman or some other bytecode fault injector :(.

[singhpk234]

since at the moment we would need custom plan rules as well I think

+1, I was also thinking in these lines, we need here analyzer rules so that we could get a DSV2Relation which hold our mocked table object (which would throw our CommitUnknowEx when commit is called). The main problem as you highlighted is the same, it supported in SQL via our Extensions :(.

[singhpk234]

@RussellSpitzer , I went one step above and used catalog to manipulate and return the spy table I want. Used static prop to manipulate to get the spy spark table instead :) !!!

[RussellSpitzer]

@singhpk234 Thats a great solution! Maybe i'l rework my PR later to do the same thing!

[rdblue]

Thanks, @singhpk234!