dynamodb batch_writer dedup on wrong key

[isunli]

Describe the bug

We use boto3 dynamodb batch writer to write items. When use overwrite_by_pkeys we got a KeyError.
Code.

with fx_rate_table.batch_writer(overwrite_by_pkeys=['FROM#TO']) as writer:
    for data in records:
        update_fx_ddb(writer, data)

Data.

{'to_currency': 'USD', 'conversion_type': 'Corporate', 'from_currency': 'CNY', 'conversion_date': '2023-08-15T00:00:00.000Z', 'conversion_rate': Decimal('0.137206207208814127'), 'FROM#TO': 'CNY#USD'}

Error.

[ERROR] KeyError: 'FROM'
Traceback (most recent call last):
  File "/var/task/fin_tech_tidal_endpoint/handlers/update_fx_rate_table.py", line 42, in update_fx_rate_table
    update_fx_ddb(writer, data)
  File "/var/task/fin_tech_tidal_endpoint/handlers/update_fx_rate_table.py", line 58, in update_fx_ddb
    writer.put_item(Item=item)
  File "/var/runtime/boto3/dynamodb/table.py", line 103, in put_item
    self._add_request_and_process({'PutRequest': {'Item': Item}})
  File "/var/runtime/boto3/dynamodb/table.py", line 110, in _add_request_and_process
    self._remove_dup_pkeys_request_if_any(request)
  File "/var/runtime/boto3/dynamodb/table.py", line 115, in _remove_dup_pkeys_request_if_any
    pkey_values_new = self._extract_pkey_values(request)
  File "/var/runtime/boto3/dynamodb/table.py", line 126, in _extract_pkey_values
    return [
  File "/var/runtime/boto3/dynamodb/table.py", line 127, in <listcomp>
    request['PutRequest']['Item'][key]

Expected Behavior

Dedup on correct key.

Current Behavior

Deduped on wrong key, our key FROM#TO got split somewhere thus returned KeyError.

Reproduction Steps

Use code/data above.

Possible Solution

No response

Additional Information/Context

No response

SDK version used

boto3-1.26.90 botocore-1.29.90

Environment details (OS name and version, etc.)

AWS Lambda - Python 3.10

[RyanFitzSimmonsAK]

Hi @isunli, thanks for reaching out. I had some trouble reproducing this key splitting behavior. Could you provide debug logs? You can get debug logs by adding boto3.set_stream_logger('') to the top of your script, and redacting any sensitive information. Thanks!

[isunli]

Thanks for your help, I couldn't reproduce either. Might be a transient error. I will reopen if see this in future. Thanks again.