Honor previous metadata maximum versions configuration defined by the user.

[shanthoosh]

Summary

The table-configuration: write.metadata.previous-versions-max in Apache Iceberg sets the maximum number of old metadata files (snapshots) to keep before they are potentially deleted after a new commit. In openhouse catalog, it is always hardcoded to be 168 regardless of the value defined for the configuration in the table-properties by the user. For streaming applications, which commits frequently (every 5 mins) to iceberg, it is essential to allow users to override this configuration so that they can control the time-travel queries for long duration and rollback to any previous snapshots if needed.

To support the above, this patch sets the configuration: write.metadata.previous-versions-max to default value of 168 only if it is not defined by the user in their table properties.

Changes

• Client-facing API Changes
• Internal API Changes
• Bug Fixes
• New Features
• Performance Improvements
• Code Style
• Refactoring
• Documentation
• Tests

For all the boxes checked, please include additional details of the changes made in this pull request.

Testing Done

Added unit tests for changes introduced.

• Manually Tested on local docker setup. Please include commands ran, and their output.
• Added new tests for the changes made.
• Updated existing tests to reflect the changes made.
• No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
• Some other form of testing like staging or soak time in production. Please explain.

For all the boxes checked, include a detailed description of the testing done for the changes made in this pull request.

Additional Information

• Breaking Changes
• Deprecations
• Large PR broken into smaller PRs, and PR plan linked in the description.

For all the boxes checked, include additional details of the changes made in this pull request.