Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

downloaders break when imported as part of another module #164

Closed
gottacatchenall opened this issue Mar 10, 2023 · 5 comments · Fixed by #167
Closed

downloaders break when imported as part of another module #164

gottacatchenall opened this issue Mar 10, 2023 · 5 comments · Fixed by #167

Comments

@gottacatchenall
Copy link
Member

gottacatchenall commented Mar 10, 2023

When building download links, this bit of code

stem = "CHELSA_$(lowercase(model_code))_r1i1p1f1_w5e5_$(scenario_code)_$(var_code)_$(month_code)_$(replace(year_sep, "-"=>"_"))_norm.tif"
can (in some cases) build a misshapen URL because it trys to inject $scenario_code where scenario_code = replace(lowercase(string(S)), "_" => "-"), and S is a SSP struct.

When I import the SDT in a module, e.g. MyModule.jl:

module MyModule
   using SpeciesDistributionToolkit
  
  function foo()
        SimpleSDMPredictor(
            RasterData(CHELSA2, BioClim),
            Projection(SSP370, GFDL_ESM4);
            layer="BIO1" ) 
  end   
   export foo
end

and run

include("MyModule.jl")
MyModule.foo()

fails because it trys to download a url with slug 2011-2040/SIMPLESDMDATASETS.GFDL-ESM4/simplesdmdatasets.ssp370/bio/CHELSA_bio1_2011-2040_simplesdmdatasets.gfdl-esm4_simplesdmdatasets.ssp370_V.2.1.tif.

@gottacatchenall
Copy link
Member Author

should be a simple fix to add

Mstr, Sstr = map(x->convert(String, split(string(x), ".")[end]), [M,S])
stem = "CHELSA_bio$(var_code)_$(year_sep)_$(lowercase(replace(Mstr, "_" => "-")))_$(lowercase(Sstr))_V.2.1.tif"

@gottacatchenall
Copy link
Member Author

gottacatchenall commented Mar 10, 2023

this is also a problem for the paths provided by destination, it creates a new dir in ENV["SDMLAYERS_PATH"] with SimpleSDMDatasets. as a prefix

@tpoisot
Copy link
Member

tpoisot commented Mar 10, 2023

Yeah, good call, it's going to do that for all providers, so I'm guessing we need an option to sanitize paths and URLs. It shouldn't be too complicated because we know exactly what string to remove.

@gottacatchenall
Copy link
Member Author

gottacatchenall commented Mar 10, 2023

makes sense. not sure if this works but if there are not other impacts it could be implemented at the string(::SSP) level

@tpoisot
Copy link
Member

tpoisot commented Apr 16, 2023

I am working on this in #167 -- I think it's "simply" a matter of catching the module name in the string, and them replacing them by an empty string.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging a pull request may close this issue.

2 participants