[FLINK-36675][table-planner] Always serialize dynamic options to exec json plan

[xuyangzhong]

What is the purpose of the change

Before this fix, dynamic option hints will not take effect when restoring exec plan from json plan without setting 'table.plan.compile.catalog-objects' to 'all'.

This fix is trying to always serialize dynamic options to exec json plan, and merge them to table options when restoring from json plan.

Brief change log

• Add a field for dynamic options in DynamicTableSpecBase to let it can be serialized to json plan
• Add UT and IT cases

Verifying this change

Some tests are added to verify this fix.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented?

[flinkbot]

CI report:

• cc21cf6 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[davidradl]

Reviewed by Chi on 05/12/24 Need a committer to review. On the face of it it looks like a good change, but we would like an expert in the are to confirm.

[github-actions]

This PR is being marked as stale since it has not had any activity in the last 90 days.
If you would like to keep this PR alive, please leave a comment asking for a review.
If the PR has merge conflicts, update it with the latest from the base branch.

If you are having difficulty finding a reviewer, please reach out to the
community, contact details can be found here: https://flink.apache.org/what-is-flink/community/

If this PR is no longer valid or desired, please feel free to close it.
If no activity occurs in the next 30 days, it will be automatically closed.

[github-actions]

This PR has been closed since it has not had any activity in 120 days.
If you feel like this was a mistake, or you would like to continue working on it,
please feel free to re-open the PR and ask for a review.