Pysail Local execution timing out

[Soumendu-Khilar]

Hello Everyone,

I am a newbie to Spark and Sail usage.

This is something that I am trying to do from my local MAC. I am trying to read a CSV file in S3 bucket which is of 15 GB size. Doing a simple read and show gives the result.
But when I try to do any other filter, I start getting timeout errors.
Encountered error while reading
While I understand that data of this volume are best to be analysed and aggregated in a cluster, I was wondering if there is still a possibility to read and do simple filters in the local system.

Data Details:
Location : S3
Size : 15 GB
Columns Size : 23

Any suggestions for optimization, if any, would really help.

[linhr]

Reading and filtering a single CSV file from remote storage requires streaming the entire file, which may take a long time depending on your network. The CSV format does not support efficient pruning.

If you have data in e.g., Parquet format, querying the data in remote storage would be much more efficient.

It is not yet clear to me whether the "timeout error" is an issue within Sail or the underlying operating system. If you can share the exact error message or a stack trace, we might be able to pinpoint the issue more clearly.

[Soumendu-Khilar]

@linhr I agree on that, a parquet format would make efficient reading. Unfortunately, most of the files that I am dealing with are CSVs.

The error message that I get is as below:
[2026-07-02T13:06:07Z INFO object_store::client::retry] Encountered transport error of kind Timeout, backing off for 0.1 seconds, retry 1 of 10: HTTP error: error sending request
[2026-07-02T13:06:29Z INFO object_store::client::get] Encountered error while reading response body: HTTP error: request or response body error. Retrying in 0.1s
[2026-07-02T13:06:31Z INFO object_store::client::get] Encountered error while reading response body: HTTP error: request or response body error. Retrying in 0.1s
[2026-07-02T13:06:35Z INFO object_store::client::get] Encountered error while reading response body: HTTP error: request or response body error. Retrying in 0.1s
[2026-07-02T13:06:35Z INFO object_store::client::get] Encountered error while reading response body: HTTP error: request or response body error. Retrying in 0.1s
[2026-07-02T13:06:36Z INFO object_store::client::get] Encountered error while reading response body: HTTP error: request or response body error. Retrying in 0.1s
[2026-07-02T13:06:37Z INFO object_store::client::get] Encountered error while reading response body: HTTP error: request or response body error. Retrying in 0.1s

I will try to look more if there is a stacktrace or any DEBUG logs that can be generated.

[Soumendu-Khilar]

@linhr Does the INFO logs give any insight of the issue, if it is on the network side or sail side?

[linhr]

You can add the environment variable RUST_LOG=sail=debug or even RUST_LOG=debug to get detailed logging. But looking at the INFO log, it seems it's a network issue specific to the data you're loading.