Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to set up JupyterHub in the first place #17

Open
filippo82 opened this issue Jun 16, 2020 · 6 comments
Open

How to set up JupyterHub in the first place #17

filippo82 opened this issue Jun 16, 2020 · 6 comments

Comments

@filippo82
Copy link

Dear @michalc,

sorry for the naive question: but how/where do I set up the JupyterHub in the first place? Does this assume that a JupyterHub (running inside a container?) is already running on AWS?

Thanks!

Best wishes,
-Filippo

@michalc
Copy link
Member

michalc commented Jun 28, 2020

Does this assume that a JupyterHub (running inside a container?) is already running on AWS?

It very almost does assume JupyterHub is already running on AWS. The spawner, which would be running in JupyterHub, extracts the private IP address of the Fargate task, and attempts to communicate to that. However...

  • Potentially you could have some sort of VPN setup with wherever JupyterHub is running and the subnet(s) in AWS, so technically JupyterHub doesn't have to be inside AWS... but typically, it would be. (I don't know too much about this approach to be honest)

  • JupyterHub itself doesn't need to be in a container, it just needs to be able to connect to the Fargate tasks (and vice versa). For example, it could be directly on an EC2 instance.

how/where do I set up the JupyterHub in the first place?

This may be not the most helpful answer, but I think I'm going to point you to Google / the JupyterHub docs, or something like https://github.com/georgebearden/aws-fargate-jupyterhub-demo or https://tljh.jupyter.org/en/latest/install/amazon.html#installing-on-amazon-web-services, otherwise, I would just be reproducing what is available elsewhere. It's probably a bit of a project whichever way you slice it...

If you get stuck on some step in terms of integration between JupyterHub and this spawner, feel free to post here/raise an issue (but realistically, I can't guarantee a reply in any timely manner)

@adpatter
Copy link

@filippo82 We were able to put together a working implementation - please let me know if we can help.

@Abhi94N
Copy link

Abhi94N commented Aug 29, 2022

Hi @adpatter. I would like to know how you managed to get a working implementation going. I'm having trouble passing spawner envars and I'm using jupyterhub version 1.4.2. This is what I'm seeing when trying to get_env() from the spawner.

>>> from jupyterhub.spawner import Spawner
>>> spawner = Spawner()
>>> spawner.get_env()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/jupyterhub/spawner.py", line 777, in get_env
    env['JUPYTERHUB_HOST'] = self.hub.public_host
AttributeError: 'NoneType' object has no attribute 'public_host'

@adpatter
Copy link

adpatter commented Aug 30, 2022

@Abhi94N The README provides guidance on how to pass environment variables to the Spawned environment
(https://github.com/uktrade/fargatespawner). I may not know what you are trying to accomplish. Please let me know if you would like to discuss over a screenshare.

Please see where environment variables are set in the containerOverrides.


c.FargateSpawner.get_run_task_args = lambda spawner: {
    'cluster': 'jupyerhub-notebooks',
    'taskDefinition': 'jupyterhub-notebook:7',
    'overrides': {
        'taskRoleArn': 'arn:aws:iam::123456789012:role/notebook-task',
        'containerOverrides': [{
            'command': spawner.cmd + [f'--port={spawner.notebook_port}', '--config=notebook_config.py'],
            'environment': [
                {
                    'name': name,
                    'value': value,
                } for name, value in spawner.get_env().items()
            ],
            'name': 'jupyterhub-notebook',
        }],
    },
    'count': 1,
    'launchType': 'FARGATE',
    'networkConfiguration': {
        'awsvpcConfiguration': {
            'assignPublicIp': 'DISABLED',
            'securityGroups': ['sg-00026fc201a4e374b'],
            'subnets':  ['subnet-01fc5f15ac710c012'],
        },
    },
}

@Abhi94N
Copy link

Abhi94N commented Aug 30, 2022

@adpatter Sure we can screenshare. So I've attempted a few things where I have tried to deploy the jupyterhub stack completely into fargate nodes with EKS as well as deploy only the single user notebook to fargate while the rest of the jupyterhub stack is deployed in a manage nodegroup.

I actually do have my envars set using the README config but I'm not able to pass envars from the hub to the spawner.

I remoted into the hub and tested the get_env() command but I was not able to get it to work from the hub itself.

@adpatter
Copy link

@Abhi94N I'm not clear on what you mean. Please email me and we can arrange a time to discuss: adpatter@umich.edu.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants