Fix Table.scan to enable case sensitive argument

[jiakai-li]

This pull request fixes below issue:

• Table scan doesn't respect casesensitive argument #1421

The change modified the DataScan._build_partition_projection method to pass case_sensitive argument when calling inclusive_projection. So that Table.scan method respects this configuration. Also added related tests.

[sungwy]

Hi @jiakai-li - thank you for reporting this issue and putting up this PR!

The behavior looks right to me, but I think it would be worth thinking through this behavior change holistically. In _DeleteFiles we also build partition projection using inclusive_projection where we only support case_sensitive=True matching.

iceberg-python/pyiceberg/table/update/snapshot.py

Lines 340 to 344 in ede363b

	def _build_partition_projection(self, spec_id: int) -> BooleanExpression:
	schema = self._transaction.table_metadata.schema()
	spec = self._transaction.table_metadata.specs()[spec_id]
	project = inclusive_projection(schema, spec)
	return project(self._predicate)

If we introduce case_sensitive=False as a behavior that can be controlled by the user on DataScan for inclusive_projection, I think its also worth thinking if we want to introduce a discrepancy in our behaviors based on API, or if we should be keeping a consistent behavior (one way or another).

Tagging @Fokko for review

[Fokko]

First of all, thanks @jiakai-li for raising this PR, and thanks @sungwy for raising that issue. And I agree, I think we should also be able to control case-sensitivity when doing deletes (defaulting to true as this is the current behavior).

[jiakai-li]

Thanks very much for the guidance guys @sungwy and @Fokko . Is it ok for me to pick up the delete part as well? I'll update this PR to include both operations if that's ok. Thanks!

[sungwy]

Yes, I think updating this PR to include the changes for both makes sense @jiakai-li 👍

Thank you again for tackling this issue!

[kevinjqliu]

Thanks for the PR! Left a comment on making the tests more readable

[jiakai-li]

Hey guys, thanks a lot for your kind guidance and great suggestions. I've updated the PR to:

• Enable Table.delete and Table.overwrite operations to control case-sensitivity
• Updated test cases to make it more readable

Please let me know if there is other changes you would like me to make, thanks.

[kevinjqliu]

added a few nit comments, generally LGTM! Thanks!

[jiakai-li]

added a few nit comments, generally LGTM! Thanks!

Thank you @kevinjqliu ! I've updated the test cases and resolved the comments. Please let me know if you have further comments.

[Fokko]

Thanks again @jiakai-li this looks very good. I've left some small comments, but in general this looks good 👍

[Fokko]

This looks great, thanks @jiakai-li for working on this, and thanks @sungwy for the sharp eye 👍