update aiida project template

[ltalirz]

Updates to the AiiDA project template

• disable data import on environment launch
• add jupyter notebook that opens on environment launch
  (with instructions on how to import & work with the data)
• various updates and bug fixes

Questions:

• @rokroskar Is there a way to set environment variables on the container level? I'm setting one (AIIDA_PATH) in the .bashrc and it is available in the terminal, but not in the jupyter notebook.

Todo:

• (optional, to decide) set up AiiDA line magic

[ltalirz]

Example to test:

Repository URL: https://github.com/ltalirz/contributed-project-templates
Archive URL: https://archive.materialscloud.org/record/file?record_id=638&filename=SSSP_1.1_PBE_efficiency.aiida&file_id=2a59c9e7-9752-47a8-8f0e-79bcdb06842c

[rokroskar]

Hi @ltalirz thanks for this update! Regarding your first question have you tried adding an export VAR=val to the post-init script?

[ltalirz]

Thanks - for some reason I didn't think of trying this.

I've modified the post-init.sh here:
https://renkulab.io/gitlab/leopold.talirz1/mc-import4/-/blob/master/post-init.sh#L39

But it turns out that doesn't propagate to the environment in the terminal

(same in the jupyter notebook)

[rokroskar]

Ah you're right of course - I thought the script was sourced but it's actually executed with bash. I think if we sourced it instead it would (probably) work...

[ltalirz]

It would be quite useful to be able to set environment variables, so I would support this.

[rokroskar]

Ok I will make an issue for it - I won't be able to get to it today, most likely...

[ltalirz]

@rokroskar this should be ready for review now. I tested it and it seems to work fine.

P.S. By the way, I again get the warning about an outdated renku version, but it displays the same version for the new one and the one of the repo

[lorenzo-cavazzi]

Thanks for the PR!
A minor note: we have a new renku-python release, you should probably include it in the template (see inline).

I tried to run the code on the notebook but I got an error, and I noticed the daemon is not running. Is this expected (i.e. I need to configure something first) or should it be running anyway?

[ltalirz]

thanks for the review @lorenzo-cavazzi !

I tried to run the code on the notebook but I got an error,

Would you mind sharing the error you saw? I remember trying it and it did work fine for me.

It may have been that you did not specify a path to an AiiDA archive file (although that field was optional, part of the template did assume that a valid URL was provided). For an example URL see #4 (comment)

The archive URL will be prefilled for users that enter via the Materials Cloud Archive, but of course the template should still work if someone creates a project from it manually and does not specify the url. I've now put in some extra guards to also cover that use case.

[ltalirz]

P.S. Yes, it is expected that the daemon is not running - I forgot to update the expected output in the README (now fixed).

[lorenzo-cavazzi]

After your commits, I don't get the exception anymore.
It's probably not relevant, but here is the error I got before:

Currently, if I don't provide the initial URL, it fails gracefully 👍

There is another minor detail on the README.md to possibly adjust, I'll provide the details in a commit suggestion. Fell free to include or discard it, then we are ready to merge this PR.

I have one final question: I noticed the repository contains uncommitted changes when I access a pristine environment. I assume a post initialization script runs something that creates those files.
Would It make sense to exclude the repo folder using the .gitignore file? Or adding some logic to the script to run also a git commit when the new files are created?
I'm not familiar with this system, so disregard my comment if the current behavior is the expected one.

[ltalirz]

I have one final question: I noticed the repository contains uncommitted changes when I access a pristine environment. I assume a post initialization script runs something that creates those files.
Would It make sense to exclude the repo folder using the .gitignore file? Or adding some logic to the script to run also a git commit when the new files are created?

Thanks! The repo folder contains the internal provenance tracking of AiiDA which I would not suggest to commit to git for the time being (for this one should think about a deeper integration between renku and AiiDA first) - however, I would prefer if users can restore it when they come back to the environment.

Will adding it to the .gitignore also lead to the contents of the folder being lost when the environment is shut down?
If yes, I suggest to keep it as is.

[lorenzo-cavazzi]

The code won't be preserved when starting a new environment from a new commit.
I personally believe it's very easy that users may add that folder by mistake to a commit if it's not ignored, but I have nothing against leaving it as is if you think it's fine. That can also be changed later if needed.

[ltalirz]

Ok!

I'm sure there will be further changes once people start using it, and we'll try to make it as user-friendly as possible.

[lorenzo-cavazzi]

Let's merge this!

[lorenzo-cavazzi]

I've tagged a new version here and included the reference in the main renku repository, so that the new templates will be included in renkulab.io on the next release SwissDataScienceCenter/renku#2114 .

If this is urgent, let us know and we can speed up the process