File CDK: Raise the default size limit of the declarative CsvDecoder

[Juan (jnr0790)]

Describe the feature request
The declarative CsvDecoder currently inherits Python's default csv.field_size_limit of 131,072 characters, which causes records with larger values to silently fail or be truncated. This issue affects users with long free-text fields (e.g., notes, descriptions, logs) in sources like Gainsight, leading to incomplete data ingestion without warning. The CSV parser used in the file-based CDK already raises this limit to 2^31 and exposes a csv_field_max_bytes parameter, but the declarative decoder does not expose or configure this limit, resulting in silent data loss.

Describe the solution you'd like
Raise the default size limit of the declarative CsvDecoder to match the file-based parser (2^31). Expose this limit as an optional configuration parameter in the CsvDecoder manifest, allowing users to set it explicitly. Additionally, implement the decoder to emit an explicit, attributable error (including stream name, field name, and observed size) when a record exceeds the limit, instead of silently truncating or failing.

Describe alternatives you've considered
Currently, the only workaround is upstream truncation or splitting of oversized fields before ingestion. No configuration exists in the CsvDecoder to adjust the limit or receive warnings when limits are exceeded. Modifying the CDK code to set the limit and improve error reporting is the preferred approach.

Additional context
This issue affects any source with long free-text fields, risking incomplete or missing data without user awareness. The problem has been confirmed with a customer and is similar to an existing issue (https://github.com/airbytehq/product-request-backlog/issues/115) related to CSV field size limits. The fix requires a code change in the CDK to align the declarative decoder's behavior with the file-based parser.

[Airbyte Support Bot (airbyte-support-bot)]

Hi Juan (@jnr0790), thank you for the detailed write-up!

We're looking into this. We've escalated it to our team for investigation and tracking: airbytehq/oncall#13293

A couple of things that would help the team scope this:

• Which CDK version were you running when you observed the truncation or failure?
• If you have a manifest snippet showing your CsvDecoder configuration and the error output you saw, pasting them inline in a fenced code block would be useful. Please mask or remove any sensitive information (API keys, passwords, tokens, connection strings, customer data) before sharing.
• Roughly how large are the field values you need to support? That helps confirm whether raising the default alone is sufficient or whether the configurable limit is also needed for your case.

---

Need more help? Join Airbyte Community Slack for peer support, or if you're a Cloud customer, open a support ticket referencing this URL.