Upgrade for compatibility with new Datasette 1.0 JSON format

[brandonrobertz]

Running datasette, version 1.0a2 this plugin seems broken.

When the search runs, it always gets undefined for the row count from data.filtered_table_rows_count.

datasette-search-all/datasette_search_all/templates/search_all.html

Line 127 in 5b5f192

var count = data.filtered_table_rows_count;

I changed this to data.count and the plugin works as it used to.

[simonw]

I also got this error:

TypeError: columns is undefined

[simonw]

I think I need a Playwright test for this, since it's a bug in the JavaScript.

I'll borrow from https://github.com/datasette/datasette-comments/blob/a0f637f6a3d6927a00c678166d7e66363df1a7b5/tests/test_ui.py#L2

[simonw]

Here's the code that's failing:

datasette-search-all/datasette_search_all/templates/search_all.html

Lines 121 to 126 in 3ecd8e6

	function displayResults(data, base_url) {
	var rows = data.rows;
	var database = data.database;
	var table = data.table;
	var columns = data.columns;
	var count = data.filtered_table_rows_count;

I don't think I've exposed all of that stuff as ?_extra= yet.

[simonw]

Actually I have: https://latest.datasette.io/fixtures/searchable.json?__search=dog&_extra=count&_extra=table&_extra=database&_extra=columns

{
  "ok": true,
  "next": null,
  "columns": [
    "pk",
    "text1",
    "text2",
    "name with . and spaces"
  ],
  "database": "fixtures",
  "count": 2,
  "table": "searchable",
  "rows": [
    {
      "pk": "pk",
      "text1": "text1",
      "text2": "text2",
      "name with . and spaces": "name with . and spaces"
    },
    {
      "pk": "pk",
      "text1": "text1",
      "text2": "text2",
      "name with . and spaces": "name with . and spaces"
    }
  ],
  "truncated": false
}

[simonw]

Found a Datasette 1.0 bug: the ?_extra=columns parameter causes all rows to return `"column-name": "column-name": https://latest.datasette.io/fixtures/attraction_characteristic.json?_extra=columns&_size=1

{
  "ok": true,
  "next": "2,2",
  "columns": [
    "pk",
    "name"
  ],
  "rows": [
    {
      "pk": "pk",
      "name": "name"
    }
  ],
  "truncated": false
}

Should return this:

{
  "ok": true,
  "next": "2,2",
  "columns": [
    "pk",
    "name"
  ],
  "rows": [
    {
      "pk": 2,
      "name": "Paranormal"
    }
  ],
  "truncated": false
}

[simonw]

That bug is a bit of a blocker on this task. For the moment I'll use the keys from the first returned row.

[simonw]

Got that working - the tests are a bit ugly in terms of Playwright boilerplate:

datasette-search-all/tests/test_playwright.py

Lines 7 to 24 in 502ac1e

	pytestmark = pytest.mark.skipif(sync_api is None, reason="playwright not installed")
	def test_ds_server(ds_server):
	with sync_api.sync_playwright() as playwright:
	browser = playwright.chromium.launch()
	page = browser.new_page()
	page.goto(ds_server + "/")
	assert page.title() == "Datasette: data"
	# It should have a search form
	assert page.query_selector('form[action="/-/search"]')
	def test_search(ds_server):
	with sync_api.sync_playwright() as playwright:
	browser = playwright.chromium.launch()
	page = browser.new_page()
	page.goto(ds_server + "/-/search?q=cleo")

Playwright includes pytest fixtures, you can use page to get a page to interact with.

I was getting "This event loop is already running" errors when I tried to use that, but it looks like the solution is to add nest_asyncio.apply() which seems to have fixed it.

• "RuntimeError: This event loop is already running" When attempting REPL example microsoft/playwright-python#178

[simonw]

Much neater:

datasette-search-all/tests/test_playwright.py

Lines 13 to 23 in e79f19a

	def test_ds_server(ds_server, page):
	page.goto(ds_server + "/")
	assert page.title() == "Datasette: data"
	# It should have a search form
	assert page.query_selector('form[action="/-/search"]')
	def test_search(ds_server, page):
	page.goto(ds_server + "/-/search?q=cleo")
	# Should show search results, after fetching them
	assert page.locator("table tr th:nth-child(1)").inner_text() == "rowid"

[simonw]

Job matrix looks pretty good!

[simonw]

Tested this on Datasette Cloud as well.