Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should you be able to upload nothing? 馃し #37

Closed
colinbrislawn opened this issue Feb 16, 2019 · 1 comment
Closed

Should you be able to upload nothing? 馃し #37

colinbrislawn opened this issue Feb 16, 2019 · 1 comment

Comments

@colinbrislawn
Copy link

colinbrislawn commented Feb 16, 2019

Steps to Reproduce:

pacifica-cli upload --proposal 000
Note how the positional argument FILES is missing.

Expected Result:

The lack of FILES argument is probably a mistake on the users part, so an error is thrown.

Actual Result:

The uploader diligently uploads an empty record. For example: https://status.datahub.pnl.gov/view/679


IDK if this is a feature or a bug. I would like to be able to upload files with zero size (representing empty samples), but uploading nothing at all seems strange to me. I'm going to put another shrugging emoji here 馃し

@dmlb2000
Copy link
Member

The idea is that the CLI uploader can be used to add metadata as a transaction if required. Empty files are problematic for an archive file system as they take metadata space without using the data space.

Archive systems often advertise how they can support 100 Petabytes of data. This space is for the data which is separate from file and directory names and their relationships (the metadata).

I understand how flag files are useful for handling state of workflow in a file system working space. However, that behavior doesn't translate well to archive systems. It would be better to use key value pair metadata to store that state and just upload nothing.

There is an open issue pacifica/pacifica-cartd#58 that discusses what it would be to add additional metadata to a download. With this feature you could use the CLI downloader to download just the metadata specified with an empty transaction. This would be useful as it allows users to have a consistent set of tooling to download and upload {meta}data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants