fix: table columns count over 3000 to save avro error#35338
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a manual Avro header writing mechanism to bypass the 64KB schema size limitation in the avro-c library, alongside improvements to NULL handling and data binding for Avro types. Feedback highlights the need to restore buffer_length assignments in TAOS_MULTI_BIND to ensure correct data processing by the TDengine client. Additionally, the reviewer pointed out potential undefined behavior in the zigzag encoding logic and recommended moving srand() out of utility functions to prevent identical sync markers and global state resets.
There was a problem hiding this comment.
Pull request overview
This PR updates taosdump Avro handling to work around large schema header creation failures and adjusts import handling for nullable binary/bytes fields.
Changes:
- Adds manual Avro object container header writing for large schemas.
- Updates Avro dump-in handling for BINARY/NCHAR/BYTES null branches and lengths.
- Adds a taosdump max-columns case to the parallel test task list.
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
tools/taos-tools/src/taosdump.c |
Adds large-schema Avro header workaround and adjusts Avro dump-in binary/bytes null/length handling. |
tests/parallel_test/cases.task |
Adds a new taosdump max-columns test case to parallel testing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Description
Issue(s)
Checklist
Please check the items in the checklist if applicable.