New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add config param for batch router for name customize #3461
Conversation
Please, review it and merge. This is very important feature. For example, adding |
Thank you @peakle for the contribution. Can you sign the CLA? |
@gitcommitshow, done |
@gitcommitshow how we move forward with this? We should wait for review from RS or something else? If we wait for review could you provide approximate deadlines for that? |
@maratbakiev2 internal disucssion has been started already. Can't say about the deadline to merge but I'll try to close the discussion before next weekend. One thing the team pointed out was that the partitioning is already supported for datalake destinations. I don't have complete knowledge about that, let me gather some more info to move the discussion ahead. |
partitioning in terms of writing to separate folders for each day is indeed supported. But! One of the most popular technologies to process that data is Hive and Hive (Hive Metastore) needs partitions in the format of |
Description
Add feature for batch router via user can add custom prefix for batch.
Notion Ticket
__
Security