JupyterHub setup

[emiliom]

Dumping some JupyterHub setup info here, for convenient access. For now, mainly from emails from Don:

Accessing our JupyterHub

• Update 11/8: https://jupyterhub.bigcz.org is now available as a user-friendly alias! But note that once you arrive at the site, the domain name devolves to the awful AWS url :(
• JupyterHub tutorial from Geohackweek 2017

Other setup info

I got the authentication to work with github, not sure if I should give other option, considering if we want to encourage people to possibly contribute, they should have a github account anyways, but it’s your call. Here are your options: https://github.com/jupyterhub/oauthenticator

I am right now working on trying to get a person environment setup after they login, this gets created and setup automatically from a Docker image. :) (note from Emilio: I think this is done?!)

I was thinking of this setup (Emilio said yes to Don's questions):

• /home/user/tutorial_contents : this is where the github repo reside, We should instruct users not to modify this too much in case of new changes, they might clash when pulling again. The user can create folder and upload their stuff to the home directory if they like. Do you think 10GB each user is enough or should I add more space for each user?
• To upload notebooks, data, etc, just simply clone repo, put stuff in appropriate folders and then push. Then user can simply pull within their jupyter notebook in the cloud. Another question is how much RAM will a user need. I don’t think there will be any gigantic dataset, so maybe 2GB ram for each user is enough?

After authentication, "check whether you have tutorial_contents folder and have the odm2client conda environment".

[emiliom]

Note to self, from Don: "You were way to quick to register into the jupyterhub, it was still in process when you logged in". So, I may need to redo things? Maybe that's why my user name is "jovyan" and not "emiliom"?

[emiliom]

Pasting these TO-DO's from @lsetiawan's email, so I don't have to track it via email 😜 :

• "I will need to add people to the whitelist, so I’ll need the core group’s github usernames"
• "could you please give me a total count of the people that will have access to the jupyterhub, so I can figure out how many instances I need?"
• setting up jupyterhub.bigcz.org subdomain (via Anthony/Stroud)

[ocefpaf]

jovyan is the default user on all Jupyter docker images

Jo·vy·an
/ˈjōvēən/
noun – an inhabitant of Jupyter

[emiliom]

jovyan is the default user on all Jupyter docker images

Thanks. I figured as much.

[lsetiawan]

Thanks Filipe! Yes, Jovyan will be everyone's username, because that the jupyterhub is doing is creating a docker container from a docker images that is built from the tutorial contents github repo Dockerfile. This why I need to know the exact number of people that will use it so I can allocate resources to scale up as more and more people register. It's a better setup and cheaper than the geohackweek setup with a very large machine that cost a lot and not very scalable, and much harder to configure. Amanda found a better way and told me to try it :) And it worked!

…

On Oct 24, 2017 12:17, "Filipe" ***@***.***> wrote: jovyan is the default user on all Jupyter docker images Jo·vy·an /ˈjōvēən/ noun – an inhabitant of Jupyter — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQ-jvItmblyaNocxwiLpVNxHouucWiLHks5svjetgaJpZM4QE2BW> .

[emiliom]

Yes, Jovyan will be everyone's username

Ok. But they're still different users, right? It's not that everyone will share one account and /home file system, and screw each other up??

This why I need to know the exact number of people that will use it so I can allocate resources to scale up as more and more people register.

Like I said over email, we won't have the exact number until next week. 20-25 people is a reasonable estimate. But I assume there's no rush to allocate resources right away, or is there an actual reason to figure that out ASAP?

It's a better setup and cheaper than the geohackweek setup with a very large machine that cost a lot and not very scalable, and much harder to configure. Amanda found a better way and told me to try it :) And it worked!

That all sounds great!

[ocefpaf]

Ok. But they're still different users, right? It's not that everyone will share one account and /home file system, and screw each other up??

I am not sure how user spaces work with jupyter but I guess that the "docker" user is always jovyan and only jovyan. I don't think the docker insteances are the same though and users files should not crash as far as I know.

[emiliom]

I am not sure how user spaces work with jupyter but I guess that the "docker" user is always jovyan and only jovyan. I don't think the docker insteances are the same though and users files should not crash as far as I know.

That's what I'm guessing too. Just want to be 100% sure.

[lsetiawan]

Ok. But they're still different users, right? It's not that everyone will share one account and /home file system, and screw each other up??

Yes, each person gets a different persistent user space of 10GB within the machine itself. Kubernetes is managing all of that. Not sure exactly the mechanic how that works. But in a point of docker it's basically creating a volume for each user, the -v /home/ocefpaf:/home/jovyan something like that I think.

Like I said over email, we won't have the exact number until next week. 20-25 people is a reasonable estimate. But I assume there's no rush to allocate resources right away, or is there an actual reason to figure that out ASAP?

No worries, based on your email I thought you wanted this ASAP especially the "self-registration", I just want the approx right now, the core people at least so I can adjust resources for that so none have trouble logging in. Thanks!

[emiliom]

Thanks. I sent you the github profiles of the core-core people last night. But it's not a problem at all if you can't get to it this week. Having @ocefpaf in the system in addition to me is already a big help.

[emiliom]

@lsetiawan FYI, the JupyterHub server went down around 10:20pm. The AWS instance is unreachable (" can’t find the server at a34cfd457bf5611e7b16c022cb2dbee7-216768378.us-west-2.elb.amazonaws.com"). It may come back on its own (I hope/assume). But I wanted to note that it went down, so you can investigate why and we can have more confidence that it won't happen again. Thanks.

[emiliom]

Just for the record: It's 1am and the JupyterHub AWS instance is back. I don't know when exactly it came back, though.

[lsetiawan]

It went down, because I took it down. Was working on it last night. Thanks.

…

On Nov 7, 2017 10:32 PM, "Emilio Mayorga" ***@***.***> wrote: @lsetiawan <https://github.com/lsetiawan> FYI, the JupyterHub server went down around 10:20pm. The AWS instance is unreachable (" can’t find the server at a34cfd457bf5611e7b16c022cb2dbee7-216768378.us-west-2.elb. amazonaws.com"). It may come back on its own (I hope/assume). But I wanted to note that it went down, so you can investigate why and we can have more confidence that it won't happen again. Thanks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQ-jvOvrEu0wb5wp1i873MXa-ZM7aDhyks5s0Ur1gaJpZM4QE2BW> .

[emiliom]

It went down, because I took it down. Was working on it last night.

Ah, that's a relief.

Next time you plan to bring it down, please ping me to let me know. Remember I often work in the evening, and the workshop is getting very close. If it happens during the day, we should let others know as well.

[lsetiawan]

Jupyterhub is currently down again. Trying to test the load. Thanks.