[docs] Add file path pattern documentation for S3 TVF and Broker Load#3337
Conversation
d1ba844 to
3df2417
Compare
There was a problem hiding this comment.
Pull request overview
This PR introduces a centralized documentation page for file path patterns (wildcards and range expansion) and wires it into the S3 TVF and Broker Load docs, plus the SQL manual sidebar.
Changes:
- Add
file-path-pattern.mdunder SQL basic elements, documenting supported URI schemes, glob-style wildcards, brace-based range expansion, examples, performance tips, and troubleshooting. - Update the S3 TVF and Broker Load manuals to describe wildcard/range support succinctly and link to the new shared documentation.
- Register the new page in
sidebars.tsunder “Basic Elements” so it appears in navigation.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
sidebars.ts |
Adds sql-manual/basic-element/file-path-pattern to the “Basic Elements” sidebar section so the new page is navigable. |
docs/sql-manual/sql-functions/table-valued-functions/s3.md |
Extends the uri parameter description and the “URI with Wildcards” section to mention wildcard and range support and link to the central file-path-pattern doc. |
docs/sql-manual/basic-element/file-path-pattern.md |
New comprehensive documentation for URI schemes, wildcard syntax, brace range expansion, usage examples (S3 TVF, Broker Load, INSERT INTO SELECT), performance considerations, and troubleshooting. |
docs/data-operate/import/import-way/broker-load-manual.md |
Documents supported file path patterns under Limitations and adds an explicit note (with link) in the HDFS wildcard example section. |
Notable issue to address before merge:
- In
file-path-pattern.md, the description of brace expansion for zero-padded numeric ranges (stating that leading zeros are stripped) conflicts with later examples that rely on fixed-width, zero-padded filenames. The behavior description and/or examples should be aligned with the actual implementation.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| | `{01..05}` | `{1,2,3,4,5}` | `1`, `2`, `3`, `4`, `5` (leading zeros stripped) | | ||
| | `{3..1}` | `{1,2,3}` | `1`, `2`, `3` (reverse ranges supported) | |
There was a problem hiding this comment.
The description for brace range {01..05} indicates that leading zeros are stripped ({1,2,3,4,5}), but later examples (e.g., month={01..12} and part-{00000..00099}.csv) rely on zero-padded values being preserved in the expanded filenames. This is internally inconsistent documentation: either the note about stripping leading zeros or the later examples are incorrect. Please update the behavior description and/or the examples so they accurately reflect how Doris actually expands zero-padded numeric ranges.
| | `{01..05}` | `{1,2,3,4,5}` | `1`, `2`, `3`, `4`, `5` (leading zeros stripped) | | |
| | `{3..1}` | `{1,2,3}` | `1`, `2`, `3` (reverse ranges supported) | | |
| | `{01..05}` | `{01,02,03,04,05}` | `01`, `02`, `03`, `04`, `05` (leading zeros preserved) | | |
| | `{3..1}` | `{3,2,1}` | `3`, `2`, `1` (reverse ranges supported) | |
314d317 to
3f15079
Compare
- Add new documentation page for file path patterns under sql-manual/basic-element
- Document supported URI formats (S3, HDFS, cloud providers)
- Document wildcard patterns (*, ?, [...]) and range expansion ({1..10})
- Add examples for S3 TVF, Broker Load, and INSERT INTO SELECT
- Include performance considerations and troubleshooting guide
- Add caution about zero-padded directory names with range patterns
- Update S3 TVF documentation to reference file-path-pattern
- Update Broker Load documentation to reference file-path-pattern
- Update INSERT INTO SELECT documentation to reference file-path-pattern
- Simplify file-analysis.md by replacing duplicate content with reference
- Add sidebar entry for new documentation
- Add Chinese translations for all documentation
- Add versioned docs for 4.x (both English and Chinese)
3f15079 to
a317ea3
Compare
Summary
sql-manual/basic-element*,?,[...]) and range expansion ({1..10})Changes
New file:
docs/sql-manual/basic-element/file-path-pattern.mdUpdated:
docs/sql-manual/sql-functions/table-valued-functions/s3.mdUpdated:
docs/data-operate/import/import-way/broker-load-manual.mdUpdated:
sidebars.tsfile-path-patternunder Basic ElementsTest plan
🤖 Generated with Claude Code