feat: Add ansi enable parameter for execution config

[comphead]

Which issue does this PR close?

• Closes feat: Introduce ansi enable parameter for execution config #18634 .

Rationale for this change

Adding a ansi flag to expand coverage for Spark built in functions, Spark 4.0 sets ansi mode as true by default.
Currently the flag is planned to be used to datafusion-spark crate via ScalarConfigArgs, however it can also be used for DF if ansi mode is in the roadmap

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

[comphead]

@Omega359 FYI

[hsiang-c]

(nit) In abs(), the value that results in overflow is returned as is.

[comphead]

Thanks @hsiang-c let me think how to rephrase it better

[hsiang-c]

LGTM

[Jefffrey]

Should we leave a note saying it has experimental support, considering it will need to be gradually introduced to functions?

Especially as I think default behaviour for some DataFusion functions lines up more with true than false (e.g. abs overflow will return compute error which is the true behaviour described here)

[comphead]

Should we leave a note saying it has experimental support, considering it will need to be gradually introduced to functions?

Especially as I think default behaviour for some DataFusion functions lines up more with true than false (e.g. abs overflow will return compute error which is the true behaviour described here)

Thanks @Jefffrey I think I overwrote those changes saying this is experimental and relevant for Spark crate only, redoing it

[Jefffrey]

Next steps would be to create an issue to retrofit this config onto existing Spark functions 👍

[comphead]

Next steps would be to create an issue to retrofit this config onto existing Spark functions 👍

Thats what @hsiang-c is working on with abs

[Omega359]

Just saw this. I was thinking of a 'spark' extension option to handle this myself.

[comphead]

Just saw this. I was thinking of a 'spark' extension option to handle this myself.

Could you please elaborate? Do you mean to have a set of spark related config options?
If so, I was thinking the same thing one day, but ansi may relate to both Spark and DataFusion IMO

[Omega359]

Just saw this. I was thinking of a 'spark' extension option to handle this myself.

Could you please elaborate? Do you mean to have a set of spark related config options? If so, I was thinking the same thing one day, but ansi may relate to both Spark and DataFusion IMO

Essentially yes. Spark seems to have a - let's put this politely - abundance of configuration options. I'm sure more than just the ansi one will apply to DF spark functions at some point.

The only real concern I have with having an ansi config option in the main df execution config is that it might start to be applied piecemeal outside of the spark crate. Think implicit casting, normal scalar udfs, etc.

I'm not against that at all but it's a concern for me because I have my doubts DF itself would (or perhaps even should?) support anything but what spark deems strict ansi mode (ansi = true). It's a topic that likely should get much more community involvement if this was to be used outside of the spark crate.