Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

base_url doesn't entirely work for running Datasette inside Binder #712

Closed
simonw opened this issue Mar 26, 2020 · 12 comments
Closed

base_url doesn't entirely work for running Datasette inside Binder #712

simonw opened this issue Mar 26, 2020 · 12 comments

Comments

@simonw
Copy link
Owner

@simonw simonw commented Mar 26, 2020

Thanks! I'm trying to launch Datasette from within a notebook using the jupyter-server-proxy and the new base_url parameter. While the assets load ok, and the breadcrumb navigation works, the facet links don't seem to use the base_url. Or have I missed something?

Originally posted by @wragge in #394 (comment)

simonw added a commit that referenced this issue Mar 26, 2020
@simonw

This comment has been minimized.

Copy link
Owner Author

@simonw simonw commented Mar 26, 2020

Weird... the unit test I just added passes. So suggested facets look like they're OK.

And yet the demo in https://mybinder.org/v2/gh/wragge/datasette-test/master?urlpath=lab doesn't work.

@simonw

This comment has been minimized.

Copy link
Owner Author

@simonw simonw commented Mar 26, 2020

Here's the truncatted HTML for https://hub.gke.mybinder.org/user/wragge-datasette-test-czqms2oy/proxy/8001/results/articles

<!DOCTYPE html>
<html>
<head>
    <title>results: articles: 400 rows
    </title>
    <link rel="stylesheet" href="/user/wragge-datasette-test-czqms2oy/proxy/8001/-/static/app.css?4434ab">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
/head>
<body class="table db-results table-articles">

<nav class="hd">
    <p class="crumbs">
        <a href="/user/wragge-datasette-test-czqms2oy/proxy/8001/">home</a> /
        <a href="/user/wragge-datasette-test-czqms2oy/proxy/8001/results">results</a>
    </p>
    
</nav>

<div class="bd">

<h1 style="padding-left: 10px; border-left: 10px solid #ff0000">articles</h1>

<h3>400 rows</h3>


<form class="filters" action="/user/wragge-datasette-test-czqms2oy/proxy/8001/results/articles" method="get">
    ...    
    <input type="submit" value="Apply">
</form>

<p><a class="not-underlined" title="select rowid, article_id, title, newspaper_id, newspaper_title, page, date, category, words, illustrated, corrections, url, page_url from articles order by rowid limit 101" href="/user/wragge-datasette-test-czqms2oy/proxy/8001/results?sql=select+rowid%2C+article_id%2C+title%2C+newspaper_id%2C+newspaper_title%2C+page%2C+date%2C+category%2C+words%2C+illustrated%2C+corrections%2C+url%2C+page_url+from+articles+order+by+rowid+limit+101">&#x270e; <span class="underlined">View and edit SQL</span></a></p>

<p class="export-links">This data as <a href="/results/articles.json">json</a>, <a href="/results/articles.csv?_size=max">CSV</a> (<a href="#export">advanced</a>)</p>
    <p class="suggested-facets">
        Suggested facets: <a href="/results/articles?_facet=newspaper_id#facet-newspaper_id">newspaper_id</a>, <a href="/results/articles?_facet=newspaper_title#facet-newspaper_title">newspaper_title</a>, <a href="/results/articles?_facet=page#facet-page">page</a>, <a href="/results/articles?_facet=category#facet-category">category</a>, <a href="/results/articles?_facet=illustrated#facet-illustrated">illustrated</a>, <a href="/results/articles?_facet=corrections#facet-corrections">corrections</a>, <a href="/results/articles?_facet_date=date#facet-date">date</a> (date)
    </p>
    <table class="rows-and-columns">
        <thead>
            <tr>
                    <th class="col-Link" scope="col">
                            Link
                    </th>
                    <th class="col-rowid" scope="col">
                                <a href="/results/articles?_sort_desc=rowid" rel="nofollow">rowid&nbsp;▼</a>
                    </th>
@simonw

This comment has been minimized.

Copy link
Owner Author

@simonw simonw commented Mar 26, 2020

I have a hunch about this: maybe the URLs that work are the ones that use {{ base_url }} directly in the template, while the ones that break are the ones that reconstruct the URL based on the incoming path?

@simonw

This comment has been minimized.

Copy link
Owner Author

@simonw simonw commented Mar 26, 2020

Here's the output of the /-/asgi-scope plugin:

{'client': ('10.12.3.15', 0),
 'headers': [(b'cookie',
              b''),
             (b'upgrade-insecure-requests', b'1'),
             (b'dnt', b'1'),
             (b'accept-encoding', b'gzip'),
             (b'accept-language', b'en-US,en;q=0.5'),
             (b'accept',
              b'text/html,application/xhtml+xml,application/xml;q=0.9,image/'
              b'webp,*/*;q=0.8'),
             (b'user-agent',
              b'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:74.0) Gecko'
              b'/20100101 Firefox/74.0'),
             (b'x-scheme', b'https'),
             (b'x-original-uri',
              b'/user/wragge-datasette-test-czqms2oy/proxy/8001/-/asgi-scope'),
             (b'x-forwarded-proto', b'https,http'),
             (b'x-forwarded-port', b'443,80'),
             (b'x-forwarded-host', b'hub.gke.mybinder.org'),
             (b'x-forwarded-for', b'148.64.98.14,10.12.3.15'),
             (b'x-real-ip', b'148.64.98.14'),
             (b'connection', b'close'),
             (b'host', b'hub.gke.mybinder.org'),
             (b'x-forwarded-context',
              b'/user/wragge-datasette-test-czqms2oy/proxy/8001'),
             (b'x-proxycontextpath',
              b'/user/wragge-datasette-test-czqms2oy/proxy/8001')],
 'http_version': '1.1',
 'method': 'GET',
 'path': '/-/asgi-scope',
 'query_string': b'',
 'raw_path': b'/-/asgi-scope',
 'root_path': '',
 'scheme': 'https,http',
 'server': ('127.0.0.1', 8001),
 'type': 'http'}

From this output... it looks like the path being passed through to Datasette is still /-/asgi-scope here when I though it would be /user/wragge-datasette-test-czqms2oy/proxy/8001/-/asgi-scope

The original URL path (the one I want) is passed in the x-original-uri HTTP header.

@simonw

This comment has been minimized.

Copy link
Owner Author

@simonw simonw commented Mar 26, 2020

Aha! It turns out my demo at https://github.com/simonw/jupyterserverproxy-datasette-demo suffers from the same bug.

@simonw

This comment has been minimized.

Copy link
Owner Author

@simonw simonw commented Mar 26, 2020

I think the solution here may be to let the incoming x-original-uri header over-ride the raw_path used by Datasette. This may require a new Datasette setting - maybe this:

--config path_from_header:x-original-uri
simonw added a commit that referenced this issue Mar 26, 2020
simonw added a commit that referenced this issue Mar 26, 2020
simonw added a commit that referenced this issue Mar 26, 2020
@simonw

This comment has been minimized.

Copy link
Owner Author

@simonw simonw commented Mar 26, 2020

I'm running this latest release like so:

Launch this binder: https://mybinder.org/v2/gh/wragge/datasette-test/master?urlpath=lab

Open the notebook.

In the notebook run this:

!pip install datasette-debug-asgi https://github.com/simonw/datasette/archive/7b205af0b53c92d17b69f8c95d70a24a0c06ebce.zip

And use this block to run Datasette:

# Get current running servers
servers = list_running_servers()

# Get the current base url
base_url = next(servers)['base_url']

# Create a base url for Datasette suing the proxy path
proxy_url = f'{base_url}proxy/8001/'

# Display a link to Datasette
display(HTML(f'<p><a href="{proxy_url}">View Datasette</a> (Click on the stop button to close the Datasette server)</p>'))

# Launch Datasette
!datasette -- results.db --port 8001 --config base_url:$proxy_url --config path_from_header:x-original-uri --config template_debug:1
@simonw simonw changed the title Suggested facet URLs don't work with base_url base_url doesn't entirely work for running Datasette inside Binder Mar 26, 2020
@simonw

This comment has been minimized.

Copy link
Owner Author

@simonw simonw commented Mar 26, 2020

Woohoo that worked!
results__articles__400_rows

@wragge

This comment has been minimized.

Copy link

@wragge wragge commented Mar 26, 2020

Great! Yes, can confirm that this works on Binder. However, when I try to run the same code locally, I get an Internal Server Error when I try to access Datasette.

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Volumes/Workspace/mycode/datasette-test/lib/python3.7/site-packages/uvicorn/protocols/http/httptools_impl.py", line 385, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/Volumes/Workspace/mycode/datasette-test/lib/python3.7/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
    return await self.app(scope, receive, send)
  File "/Volumes/Workspace/mycode/datasette-test/lib/python3.7/site-packages/datasette_debug_asgi.py", line 24, in wrapped_app
    await app(scope, recieve, send)
  File "/Volumes/Workspace/mycode/datasette-test/lib/python3.7/site-packages/datasette/utils/asgi.py", line 174, in __call__
    await self.app(scope, receive, send)
  File "/Volumes/Workspace/mycode/datasette-test/lib/python3.7/site-packages/datasette/tracer.py", line 75, in __call__
    await self.app(scope, receive, send)
  File "/Volumes/Workspace/mycode/datasette-test/lib/python3.7/site-packages/datasette/app.py", line 746, in __call__
    raw_path = dict(scope["headers"])[path_from_header.encode("utf8")].split(b"?")[0]
KeyError: b'x-original-uri'
INFO:     127.0.0.1:49320 - "GET / HTTP/1.1" 500 Internal Server Error
@wragge

This comment has been minimized.

Copy link

@wragge wragge commented Mar 26, 2020

Following on from @betatim's suggestion on Twitter, I've changed the proxy url to include 'absolute'.

proxy_url = f'{base_url}proxy/absolute/8001/'

This works both on Binder and locally, without using the path_from_header option. I've updated the demo repository. Sorry @simonw if I've led you down the wrong path!

@simonw

This comment has been minimized.

Copy link
Owner Author

@simonw simonw commented Mar 26, 2020

Hooray! This is a really great fix, thanks.

@simonw simonw closed this Mar 26, 2020
@simonw

This comment has been minimized.

Copy link
Owner Author

@simonw simonw commented Mar 26, 2020

And for https://github.com/simonw/jupyterserverproxy-datasette-demo setting "absolute_url": True appears to fix the bug as well (it previously caused an infinite redirect loop but that seems not to happen any more with the latest base_url setting): simonw/jupyterserverproxy-datasette-demo@880f9d5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.