Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better file handling #108

Open
rvosa opened this issue May 3, 2024 · 0 comments
Open

Better file handling #108

rvosa opened this issue May 3, 2024 · 0 comments

Comments

@rvosa
Copy link
Member

rvosa commented May 3, 2024

The pipeline operates on a SQLite database file that is progressively populated by a chain of rules. Because SnakeMake gets confused about time stamps and rule dependencies when the same file is modified by multiple rules, separate (0-byte) files are created. This is messy. Supposedly, the SnakeMake team is working on facilitating the SQLite use case in upcoming releases. It would be good if the pipeline could be made cleaner in that way. In addition, the handling of other files should also be improved. Users paint themselves into corners with poorly configured run attempts and then don't know what to remove to try again. This should be more fool-proof, possibly simply by adding a clean rule. Even better would be if not all intermediate results were lost, although that's harder to do correctly given certain failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Todo
Development

No branches or pull requests

1 participant