Skip to content

Bug: append_datasource hardcodes multipart form field name to 'csv', breaking Parquet and NDJSON file uploads #23

Description

@zodiac12k

Description

When using append_datasource to upload a local .parquet or .ndjson file, the Tinybird API rejects the request with the following error:

NDJSON/Parquet multipart requests require name field set to 'ndjson'/'parquet'. Example: curl -F "ndjson=@events.ndjson" ...

Root Cause

In tinybird_sdk/api/api.py (around line 275 in v0.4.0), the multipart form field name is strictly hardcoded to "csv", regardless of the actual file format being uploaded:

# tinybird_sdk/api/api.py
            with open(file_path_str, "rb") as fp:
                file_content = fp.read()
            content_type, multipart = create_multipart_body(
                files=[("csv", file_path_str, file_content, None)],  # 🐛 BUG: hardcoded to "csv"
            )

Even though detect_data_format(source_ref) correctly sets query["format"] = "parquet", the Tinybird API expects the multipart field name to be "parquet" as well, causing a schema validation failure on the server side.

Steps to Reproduce

from tinybird_sdk import create_tinybird_api

api = create_tinybird_api({"token": "your_token", "base_url": "https://api.tinybird.co"})

# Attempting to upload a valid parquet file
api.append_datasource(
    "krx_daily_stocks",
    {"file": "/path/to/data.parquet"}
)
# Raises TinybirdApiError: NDJSON/Parquet multipart requests require name field set to 'ndjson'/'parquet'.

Proposed Fix

The field name in create_multipart_body should be dynamically set based on the detected_format (e.g., "csv", "parquet", "ndjson").

            # Proposed Fix
            form_field_name = detected_format if detected_format else "csv"
            content_type, multipart = create_multipart_body(
                files=[(form_field_name, file_path_str, file_content, None)],
            )

Environment

  • SDK Version: 0.4.0
  • Python Version: 3.13

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions