Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a decompressor for tar files #38

Closed
pnuu opened this issue Sep 25, 2019 · 11 comments

Comments

@pnuu
Copy link
Contributor

commented Sep 25, 2019

Currently the available on-the-fly decompression methods are bzip2 and xrit. Decompressor for .tar, .tar.gz, .tgz and .tar.bz2 files is needed. The tarfile package seems to be a good candidate for this.

@pnuu pnuu added the enhancement label Sep 25, 2019
@pnuu pnuu self-assigned this Sep 25, 2019
@mraspaud

This comment has been minimized.

Copy link
Member

commented Sep 25, 2019

There is an untaring feature in the move_it_server...

@pnuu

This comment has been minimized.

Copy link
Contributor Author

commented Sep 25, 2019

Is there? I went through server.py before creating this issue and found only those two decompressors.

@mraspaud

This comment has been minimized.

Copy link
Member

commented Sep 25, 2019

@mraspaud

This comment has been minimized.

Copy link
Member

commented Sep 25, 2019

It actually makes more sense to unpack after transfering the file. However, I think what we have now just handles .tar, not the zipped versions of if.

@mraspaud

This comment has been minimized.

Copy link
Member

commented Sep 25, 2019

oh, looks like it actually does: Open for reading with transparent compression (recommended).

@pnuu

This comment has been minimized.

Copy link
Contributor Author

commented Sep 25, 2019

Yes, was just about to comment that the unpacking in tarfile works by default for all the versions I listed above.

So maybe we should move all the other decompressors on the client side?

@mraspaud

This comment has been minimized.

Copy link
Member

commented Sep 25, 2019

ah, but we would need to add a couple more extension to this: https://github.com/pytroll/trollmoves/blob/master/trollmoves/client.py#L211

@pnuu

This comment has been minimized.

Copy link
Contributor Author

commented Sep 25, 2019

Isn't that just the name of the "unpacker" in the config?

@mraspaud

This comment has been minimized.

Copy link
Member

commented Sep 25, 2019

it's also used to check the ending of the file: https://github.com/pytroll/trollmoves/blob/master/trollmoves/client.py#L264

@mraspaud

This comment has been minimized.

Copy link
Member

commented Sep 25, 2019

indeed, we should have all decompressors available on the client side, but I'm not convinced we should remove them from the server side. Could it be that we sometime don't really have control over the client side ?

@pnuu

This comment has been minimized.

Copy link
Contributor Author

commented Sep 25, 2019

Agreed. I'll create another ticket for the unpack'ability on both client and server sides, and close this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.