[docs] Update quickstart-flink doc to use kv.snapshot.interval 0s to avoid client obtain token issue#2660
Conversation
…avoid client obtain token issue
28a5778 to
0249270
Compare
| s3.secret-key: rustfsadmin | ||
| s3.path-style-access: true | ||
| kv.snapshot.interval: 60s | ||
| kv.snapshot.interval: 0s |
There was a problem hiding this comment.
Unfortunately, we'll still need it. Otherwise, it'll throw exception
org.apache.fluss.exception.SecurityTokenException: Failed to get file access security token: The security token included in the request is invalid. (Service: AWSSecurityTokenService; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: fabc25ae-dffb-43bf-af19-f2896f00d918; Proxy: null)
when client try to get sts token from server.
Similar to #2148
But to support it, we may need change the code the generate sts token in s3 filesystem.
See:
https://github.com/orgs/rustfs/discussions/1239
There was a problem hiding this comment.
@luoyuxia Thank you, could you open an issue in fluss to fix the way to generate sts token for s3?
| datalake.iceberg.s3.path-style-access: true | ||
| datalake.iceberg.client.region: us-east-1 | ||
| volumes: | ||
| - ./lib/fluss-lake-iceberg-$FLUSS_DOCKER_VERSION$.jar:/opt/fluss/plugins/iceberg/fluss-lake-iceberg-$FLUSS_DOCKER_VERSION$.jar |
There was a problem hiding this comment.
we don't need to mount the iceberg plugin jar
| s3.secret-key: rustfsadmin | ||
| s3.path-style-access: true | ||
| kv.snapshot.interval: 60s | ||
| kv.snapshot.interval: 0s |
There was a problem hiding this comment.
@luoyuxia Thank you, could you open an issue in fluss to fix the way to generate sts token for s3?
website/docs/quickstart/flink.md
Outdated
|
@luoyuxia did you verify if this is an issue in Flink QuickStart? I was able to make this work with Flink QuickStart. It was only the paimon/iceberg one that was failing |
|
@wuchong Thanks for review. Comments addressed. |
If just follow the steps of Flink QuickStart. I won't meet this problem. But if I sumibit a sql to query all data of a primary key table, the issue will hapeen |
I see, maybe not as part of this PR, let's also try to add specific SQL query that cause the issue in? So that when we revisit this after fixing, we can naturally validate by running the additional query. Thank you for catching! |
… to avoid client obtain token issue (#2660)
Purpose
Linked issue: close #xxx
To fix client obtain token issue
Brief change log
Tests
API and Format
Documentation