Skip to content

[docs] Update quickstart-flink doc to use kv.snapshot.interval 0s to avoid client obtain token issue #2660

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
wuchong merged 2 commits into from
Feb 12, 2026
Merged

[docs] Update quickstart-flink doc to use kv.snapshot.interval 0s to avoid client obtain token issue #2660

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0249270
[docs] Update quickstart-flink doc to use kv.snapshot.interval 0s to …
luoyuxia Feb 12, 2026
2dfe64c
address comments
luoyuxia Feb 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 1 addition & 23 deletions website/docs/quickstart/flink.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ services:
s3.access-key: rustfsadmin
s3.secret-key: rustfsadmin
s3.path-style-access: true
kv.snapshot.interval: 60s
kv.snapshot.interval: 0s
Copy link
Contributor Author

@luoyuxia luoyuxia Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, we'll still need it. Otherwise, it'll throw exception

org.apache.fluss.exception.SecurityTokenException: Failed to get file access security token: The security token included in the request is invalid. (Service: AWSSecurityTokenService; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: fabc25ae-dffb-43bf-af19-f2896f00d918; Proxy: null)

when client try to get sts token from server.
Similar to #2148

But to support it, we may need change the code the generate sts token in s3 filesystem.
See:
https://github.com/orgs/rustfs/discussions/1239

Copy link
Member

@wuchong wuchong Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@luoyuxia Thank you, could you open an issue in fluss to fix the way to generate sts token for s3?

Copy link
Contributor Author

@luoyuxia luoyuxia Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

create in #2661

zookeeper:
restart: always
image: zookeeper:3.9.2
Expand Down Expand Up @@ -161,7 +161,6 @@ volumes:
The Docker Compose environment consists of the following containers:
- **RustFS:** an S3-compatible object storage for tiered storage. You can access the RustFS console at http://localhost:9001 with credentials `rustfsadmin/rustfsadmin`. An init container (`rustfs-init`) automatically creates the `fluss` bucket on startup.
- **Fluss Cluster:** a Fluss `CoordinatorServer`, a Fluss `TabletServer` and a `ZooKeeper` server.
- Snapshot interval `kv.snapshot.interval` is configured as 60 seconds. You may want to configure this differently for production systems
- Credentials are configured directly with `s3.access-key` and `s3.secret-key`. Production systems should use CredentialsProvider chain specific to cloud environments.
- **Flink Cluster**: a Flink `JobManager`, a Flink `TaskManager`, and a Flink SQL client container to execute queries.

Expand Down Expand Up @@ -452,27 +451,6 @@ The following command allows you to quit Flink SQL Client.
quit;
```

### Remote Storage

Finally, you can use the following command to view the Primary Key Table snapshot files stored on RustFS:

```shell
docker run --rm --net=host \
-e MC_HOST_rustfs=http://rustfsadmin:rustfsadmin@localhost:9000 \
minio/mc ls --recursive rustfs/fluss/
```

Sample output:
```shell
[2026-02-03 20:28:59 UTC] 26KiB STANDARD remote-data/kv/fluss/enriched_orders-3/0/shared/4f675202-e560-4b8e-9af4-08e9769b4797
[2026-02-03 20:27:59 UTC] 11KiB STANDARD remote-data/kv/fluss/enriched_orders-3/0/shared/87447c34-81d0-4be5-b4c8-abcea5ce68e9
[2026-02-03 20:28:59 UTC] 0B STANDARD remote-data/kv/fluss/enriched_orders-3/0/snap-0/
[2026-02-03 20:28:59 UTC] 1.1KiB STANDARD remote-data/kv/fluss/enriched_orders-3/0/snap-1/_METADATA
[2026-02-03 20:28:59 UTC] 211B STANDARD remote-data/kv/fluss/enriched_orders-3/0/snap-1/aaffa8fc-ddb3-4754-938a-45e28df6d975
[2026-02-03 20:28:59 UTC] 16B STANDARD remote-data/kv/fluss/enriched_orders-3/0/snap-1/d3c18e43-11ee-4e39-912d-087ca01de0e8
[2026-02-03 20:28:59 UTC] 6.2KiB STANDARD remote-data/kv/fluss/enriched_orders-3/0/snap-1/ea2f2097-aa9a-4c2a-9e72-530218cd551c
```

## Clean up
After finishing the tutorial, run `exit` to exit Flink SQL CLI Container and then run
```shell
Expand Down
1 change: 0 additions & 1 deletion website/docs/quickstart/lakehouse.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -342,7 +342,6 @@ services:
datalake.iceberg.s3.path-style-access: true
datalake.iceberg.client.region: us-east-1
volumes:
- ./lib/fluss-lake-iceberg-$FLUSS_DOCKER_VERSION$.jar:/opt/fluss/plugins/iceberg/fluss-lake-iceberg-$FLUSS_DOCKER_VERSION$.jar
Copy link
Contributor Author

@luoyuxia luoyuxia Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't need to mount the iceberg plugin jar

- ./lib/iceberg-aws-1.10.1.jar:/opt/fluss/plugins/iceberg/iceberg-aws-1.10.1.jar
- ./lib/iceberg-aws-bundle-1.10.1.jar:/opt/fluss/plugins/iceberg/iceberg-aws-bundle-1.10.1.jar
- ./lib/postgresql-42.7.4.jar:/opt/fluss/plugins/iceberg/postgresql-42.7.4.jar
Expand Down