Skip to content

Conversation

@lizhou1111
Copy link
Contributor

@lizhou1111 lizhou1111 commented Jan 11, 2024

PR checklist:

  • Did you run ClangFormat ?
  • Did you separate headers to a different section in existing community code base ?
  • Did you surround proton: starts/ends for new code in existing community code base ?

Please write user-readable short description of the changes:
this closed #469
Originnal file storage supports streaming query, but the global aggregation is triggered with an interval.

SELECT
  count()
FROM
  file('/home/timeplus/lijianan/ssd/protondata/server_data/user_files/data', 'CSV', 'Time string, Exchange string, Symbol string, Sale_Condition string, Trade_Volume uint32, Trade_Price decimal(20,6), Trade_Stop_Stock_Indicator string, Trade_Correction_Indicator string, Sequence_Number uint32, Trade_Id uint64, Source_of_Trade string, Trade_Reporting_Facility string, Participant_Timestamp string, Trade_Reporting_Facility_TRF_Timestamp string, Trade_Through_Exempt_Indicator uint8')
SETTINGS
  format_csv_delimiter = '|', input_format_parallel_parsing = 0

Query id: cc2652fd-6025-4772-8bc5-e75597d4e087

Connecting to localhost:8463 as user default.
Connected to proton server version 1.3.31 revision 54459.

┌─explain─────────────────────────┐
│ (Expression)                    │
│ ExpressionTransform             │
│   (StreamingAggregating)        │
│   GlobalAggregatingTransform    │
│     (Expression)                │
│     ExpressionTransform         │
│       (WatermarkStep)           │
│       WatermarkStamperTransform │
│         (ReadFromStorage)       │
│         File 01              │
└─────────────────────────────────┘

@lizhou1111 lizhou1111 marked this pull request as ready for review January 11, 2024 02:10
@lizhou1111 lizhou1111 self-assigned this Jan 11, 2024
@lizhou1111 lizhou1111 requested a review from yl-lisen January 11, 2024 02:10
Copy link
Collaborator

@yl-lisen yl-lisen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yl-lisen yl-lisen merged commit f56f58f into develop Jan 16, 2024
@yl-lisen yl-lisen deleted the bugfix/issue-469-make-file-storage-unsupport-streaming-query branch January 16, 2024 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

count() on file stream returns different result than view on it

3 participants