Can not use TqdmCallback as childCallback for ArrowFSWrapper get

[OneSizeFitsQuorum]

Currently, I am overriding TqdmCallback to support showing both directory-level and file-level progress when using the get method. However, I noticed that after this PR (35da646), the opened files are no longer seekable. This results in the function for getting the file size becoming non-callable, which in turn prevents supporting progress tracking at the single file level.

I would like to know why the seekable flag is being forcibly set to False here. If this behavior is necessary, should there be an option to set it to True?

[martindurant]

I believe the PR you reference if actually setting the value of seekable to True by default ( diff ). Are you indeed getting unseekable files?

[OneSizeFitsQuorum]

I mean the code here

[OneSizeFitsQuorum]

@martindurant hi, could you please review this issue again~

[martindurant]

I would like to know why the seekable flag is being forcibly set to False here.

OK, I finally understood what you were referencing, sorry.

I think the reason for this, is that the seekable variety uses more resources (e.g., an initial lookup to determine the file-size!), so this is more efficient. I think it would be reasonable to make this a kwarg, so that you can set seekable=True for the case of wanting better feedback. In fact, you could benchmark if really it makes a difference.

[OneSizeFitsQuorum]

@martindurant Has submitted a pr to fix this