Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add gitfile option to make it possible to use local git repos when importing modules #1376

Conversation

Smeds
Copy link
Contributor

@Smeds Smeds commented Feb 4, 2022

Feat: add gitfile option to make it possible to use local git repos when importing modules.

Should make it possible to include module like this:

module dna_seq:
    snakefile: gitfile("file:///beegfs-storage/LOCAL_PATH/dna-seq-gatk-variant-calling", path="workflow/Snakefile", tag="main")
    config: config

solves issues:

@Smeds Smeds changed the title Support for local modules using git feat: add gitfile option to make it possible to use local git repos when importing modules Feb 7, 2022
@Smeds Smeds force-pushed the add_fix-support-for-module-with-localgitfile branch from e5f5208 to aa628e9 Compare February 7, 2022 10:43
@Smeds Smeds force-pushed the add_fix-support-for-module-with-localgitfile branch 2 times, most recently from b4e11f0 to 1a4491a Compare February 18, 2022 15:24
Copy link
Contributor

@johanneskoester johanneskoester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot. Some comments below. Apart from that, can you add some test case(s)?

snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
@Smeds Smeds force-pushed the add_fix-support-for-module-with-localgitfile branch 2 times, most recently from 0dcbd17 to dd47a78 Compare March 17, 2022 09:00
@Smeds Smeds force-pushed the add_fix-support-for-module-with-localgitfile branch from dd47a78 to 2fa3791 Compare March 17, 2022 09:08
@Smeds
Copy link
Contributor Author

Smeds commented Mar 17, 2022

Thanks a lot. Some comments below. Apart from that, can you add some test case(s)?

I have updated the code now.

I could add small git repo in the tests folder with commits that a workflow will have to use.

@Smeds
Copy link
Contributor Author

Smeds commented Mar 21, 2022

I need some thoughts of how to handle tests for this pull-request. I have create "module" use to test the new functionality:
1 - A successfull use of gitfile
2 - A case where a rule is missing
3 - A case where a schema is missing

But to be able to run the test I would have to include a .git folder with the module, which git isn't alllowing me to do.

The only solution I can think of now is to compress the folder and decompress it before running the different tests.

@johanneskoester Do you have any good suggestion of how to handle this?

@Smeds
Copy link
Contributor Author

Smeds commented Mar 21, 2022

Could git Submodules be a acceptable solution?

@Smeds
Copy link
Contributor Author

Smeds commented Mar 22, 2022

Could git Submodules be a acceptable solution?

Didn't work

@Smeds
Copy link
Contributor Author

Smeds commented Mar 29, 2022

)

I have added test cases now

snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
Copy link
Contributor

@johanneskoester johanneskoester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are getting there! See below.

snakemake/sourcecache.py Outdated Show resolved Hide resolved
snakemake/sourcecache.py Outdated Show resolved Hide resolved
@Smeds
Copy link
Contributor Author

Smeds commented Apr 25, 2022

@johanneskoester would there be any point of supporting relative path when importing a module using gitfile? At the moment I believe the could requires an absolute path.

@Smeds Smeds force-pushed the add_fix-support-for-module-with-localgitfile branch from 078f7bb to 0893bf3 Compare April 25, 2022 09:51
@sonarcloud
Copy link

sonarcloud bot commented Apr 25, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.9% 0.9% Duplication

@Smeds
Copy link
Contributor Author

Smeds commented May 5, 2022

done

@sonarcloud
Copy link

sonarcloud bot commented Jun 27, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.8% 0.8% Duplication

@Smeds
Copy link
Contributor Author

Smeds commented Aug 22, 2022

@johanneskoester it should now allow for relative paths

@sonarcloud
Copy link

sonarcloud bot commented Aug 22, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.6% 0.6% Duplication

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants