Can't find instructions for installing repository for development + jinja2.exceptions.TemplateNotFound: index.html

[MarcSkovMadsen]

🐛 Bug

I've tried searching several times on github, https://lightning.ai/. But I simply cannot find a guide on installing this project for development. I want to contribute a PanelFrontend - see #13335

To Reproduce

Checkout https://github.com/Lightning-AI/lightning. Most other repositories I know have an easy to find link to the instructions for installing for development.

Checkout https://lightning.ai/ and https://pytorch-lightning.readthedocs.io/en/latest/generated/CONTRIBUTING.html. I am not able to find clear instructions on installing.

Expected behavior

I would expect it to be very easy to find clear and simple instructions.

Additional Context - jinja2.exceptions.TemplateNotFound: index.html

I am working on PR #13531 which I now want to run. But I cant. To reproduce

git clone https://github.com/MarcSkovMadsen/lightning.git
git checkout b59f80224843886459d54c828325683d770da746
cd lightning
python -m venv .venv
source .venv/Scripts/activate
pip install -e .

And then run

$ lightning run app docs/source-app/workflows/add_web_ui/panel/examples/app_basic.py
Your Lightning App is starting. This won't take long.
INFO: Your app has started. View it in your browser: http://127.0.0.1:7501/view
template name index.html
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "C:\repos\private\lightning\.venv\lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 372, in run_asgi  
    result = await app(self.scope, self.receive, self.send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 75, in __call__        
    return await self.app(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\fastapi\applications.py", line 269, in __call__
    await super().__call__(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starsessions\middleware.py", line 85, in __call__
    await self.app(scope, receive, send_wrapper)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\middleware\cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\exceptions.py", line 93, in __call__
    raise exc
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\exceptions.py", line 82, in __call__
    await self.app(scope, receive, sender)
  File "C:\repos\private\lightning\.venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__       
    raise e
  File "C:\repos\private\lightning\.venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__       
    await self.app(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\routing.py", line 670, in __call__
    await route.handle(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\routing.py", line 266, in handle
    await self.app(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\routing.py", line 65, in app
    response = await func(request)
  File "C:\repos\private\lightning\.venv\lib\site-packages\fastapi\routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "C:\repos\private\lightning\.venv\lib\site-packages\fastapi\routing.py", line 160, in run_endpoint_function
    return await dependant.call(**values)
  File "c:\repos\private\lightning\src\lightning_app\core\api.py", line 258, in frontend_route
    return templates.TemplateResponse("index.html", {"request": request})
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\templating.py", line 97, in TemplateResponse
    template = self.get_template(name)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\templating.py", line 84, in get_template
    return self.env.get_template(name)
  File "C:\repos\private\lightning\.venv\lib\site-packages\jinja2\environment.py", line 997, in get_template
    return self._load_template(name, globals)
  File "C:\repos\private\lightning\.venv\lib\site-packages\jinja2\environment.py", line 958, in _load_template
    template = self.loader.load(self, name, self.make_globals(globals))
  File "C:\repos\private\lightning\.venv\lib\site-packages\jinja2\loaders.py", line 125, in load
    source, filename, uptodate = self.get_source(environment, name)
  File "C:\repos\private\lightning\.venv\lib\site-packages\jinja2\loaders.py", line 214, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: index.html
template name index.html
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "C:\repos\private\lightning\.venv\lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 372, in run_asgi  
    result = await app(self.scope, self.receive, self.send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 75, in __call__        
    return await self.app(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\fastapi\applications.py", line 269, in __call__
    await super().__call__(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starsessions\middleware.py", line 85, in __call__
    await self.app(scope, receive, send_wrapper)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\middleware\cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\exceptions.py", line 93, in __call__
    raise exc
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\exceptions.py", line 82, in __call__
    await self.app(scope, receive, sender)
  File "C:\repos\private\lightning\.venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__       
    raise e
  File "C:\repos\private\lightning\.venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__       
    await self.app(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\routing.py", line 670, in __call__
    await route.handle(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\routing.py", line 266, in handle
    await self.app(scope, receive, send)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\routing.py", line 65, in app
    response = await func(request)
  File "C:\repos\private\lightning\.venv\lib\site-packages\fastapi\routing.py", line 227, in app
    raw_response = await run_endpoint_function(
  File "C:\repos\private\lightning\.venv\lib\site-packages\fastapi\routing.py", line 160, in run_endpoint_function
    return await dependant.call(**values)
  File "c:\repos\private\lightning\src\lightning_app\core\api.py", line 258, in frontend_route
    return templates.TemplateResponse("index.html", {"request": request})
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\templating.py", line 97, in TemplateResponse
    template = self.get_template(name)
  File "C:\repos\private\lightning\.venv\lib\site-packages\starlette\templating.py", line 84, in get_template
    return self.env.get_template(name)
  File "C:\repos\private\lightning\.venv\lib\site-packages\jinja2\environment.py", line 997, in get_template
    return self._load_template(name, globals)
  File "C:\repos\private\lightning\.venv\lib\site-packages\jinja2\environment.py", line 958, in _load_template
    template = self.loader.load(self, name, self.make_globals(globals))
  File "C:\repos\private\lightning\.venv\lib\site-packages\jinja2\loaders.py", line 125, in load
    source, filename, uptodate = self.get_source(environment, name)
  File "C:\repos\private\lightning\.venv\lib\site-packages\jinja2\loaders.py", line 214, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: index.html

[MarcSkovMadsen]

Maybe the concrete issue I have is that the index.html file has been moved to the public subfolder?

UPDATE: I am issing the lightning_app/ui folder. Its probably build by something I don't know about.

[MarcSkovMadsen]

Please note I am on a windows laptop in a security constrained enterprise environment. I prefer pip but can use conda.

[MarcSkovMadsen]

WTF :-) It seems the setup.cfg has some instructions for installing and building.

But when I run export PACKAGE_NAME=app ; python setup.py sdist bdist_wheel I can see it does not include the ui folder that is missing.

[awaelchli]

For development, I usually had to download the frontend manually by running python scripts/download_frontend.py. It looks like this helper script was not yet ported over to the OSS for lightning. We need to include it or automate this step on installation. I'm pretty sure this is the reason for the template not found error.

[MarcSkovMadsen]

Thanks @awaelchli . I cannot run that command as scripts is not included in the lightning repository. Do you have another suggestion? Thanks.

[tchaton]

@MarcSkovMadsen Could you try again with PACKAGE_NAME=app pip install -e .?

Here is the function to be executed to download the UI: https://github.com/Lightning-AI/lightning/blob/master/.actions/setup_tools.py#L330

[hhsecond]

Update: Automated UI download on pip install has been introduced in #13499

[stale]

This issue has been automatically marked as stale because it hasn't had any recent activity. This issue will be closed in 7 days if no further activity occurs. Thank you for your contributions - the Lightning Team!