Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support timestamp comparison when writing data to Redis #153

Open
yunfengzhou-hub opened this issue Jun 5, 2023 · 0 comments
Open

Support timestamp comparison when writing data to Redis #153

yunfengzhou-hub opened this issue Jun 5, 2023 · 0 comments
Assignees
Labels
type:feature New feature or request

Comments

@yunfengzhou-hub
Copy link
Collaborator

FlinkProcessor's RedisSink should support writing features with timestamp to Redis, where only features with larger keys preserve in Redis.

This function has not been supported so far because in a Redis cluster, the timestamp of a feature and the feature itself might be distributed onto different Redis nodes, and querying across nodes might bring a performance overhead. It is recently discovered that Redis cluster supports grouping keys with hashtags, and key-value pairs with the same hashtag would be held by the same cluster node. Thus this performance overhead can be walked around now.

@yunfengzhou-hub yunfengzhou-hub self-assigned this Jun 5, 2023
@yunfengzhou-hub yunfengzhou-hub added comp:connector type:improvement improvement to existing functionality type:feature New feature or request and removed type:improvement improvement to existing functionality comp:connector labels Sep 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:feature New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant