Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Columns starting with an underscore behave poorly in filters #1527

Closed
simonw opened this issue Nov 22, 2021 · 7 comments
Closed

Columns starting with an underscore behave poorly in filters #1527

simonw opened this issue Nov 22, 2021 · 7 comments
Labels

Comments

@simonw
Copy link
Owner

simonw commented Nov 22, 2021

Similar bug to #1525 (and #1506 before it). Start on https://latest.datasette.io/fixtures/facetable?_facet=_neighborhood - then select a neighborhood - then try to remove that filter using the little "x" and submitting the form again.

filter-bug

@simonw simonw added the bug label Nov 22, 2021
@simonw
Copy link
Owner Author

simonw commented Nov 22, 2021

I think the root cause is this hidden form field on https://latest.datasette.io/fixtures/facetable?_facet=_neighborhood&_neighborhood__exact=Downtown

            <input type="hidden" name="_facet" value="_neighborhood">
            <input type="hidden" name="_neighborhood__exact" value="Downtown">
        <input type="submit" value="Apply">

@simonw
Copy link
Owner Author

simonw commented Nov 30, 2021

I started fiddling with a test for this which extracts the <select> options and ALL of the <input> fields, but I probably won't use it:

def test_exact_parameter_results_in_correct_hidden_fields(app_client):
    # https://github.com/simonw/datasette/issues/1527
    response = app_client.get(
        "/fixtures/facetable?_facet=_neighborhood&_neighborhood__exact=Downtown"
    )
    # In this case we should NOT have a hidden _neighborhood__exact=Downtown field
    form = Soup(response.body, "html.parser").find("form")
    selects = [
        {
            "name": select["name"],
            "value": select.select("option[selected]")[0].text
            if select.select("option[selected]")
            else "",
        }
        for select in form.findAll("select")
    ]
    inputs = [input.attrs for input in form.findAll("input")]
    # Turn those both into a {name: (value, type)} array
    form_inputs = {}
    form_inputs.update(
        {select["name"]: (select["value"], "select") for select in selects}
    )
    form_inputs.update(
        {
            input["name"]: (input.get("value"), input["type"])
            for input in inputs
            if input.get("name")
        }
    )
    assert form_inputs == {
        "_filter_column_1": ("_neighborhood", "select"),
        "_filter_op_1": ("=", "select"),
        "_filter_value_1": ("Downtown", "text"),
        "_filter_column": ("", "select"),
        "_filter_op": ("", "select"),
        "_filter_value": (None, "text"),
        "_sort": ("Sort by pk", "select"),
        "_sort_by_desc": (None, "checkbox"),
        "_facet": ("_neighborhood", "hidden"),
        "_neighborhood__exact": ("Downtown", "hidden"),
    }

The problem is that last hidden field, _neighborhood__exact=Downtown - which should not be there.

@simonw
Copy link
Owner Author

simonw commented Nov 30, 2021

Manually tested this too, looks like that fixed it.

@simonw simonw closed this as completed Nov 30, 2021
simonw added a commit that referenced this issue Nov 30, 2021
@simonw simonw changed the title Columns starting with an underscore can't be removed from filters Columns starting with an underscore behave poorly in filters Dec 7, 2021
@simonw
Copy link
Owner Author

simonw commented Dec 7, 2021

Found a new case of this bug: click the "Apply" button on https://latest.datasette.io/fixtures/facetable?_sort=pk&_city_id__gt=1

apply-bug

@simonw simonw reopened this Dec 7, 2021
@simonw simonw added this to the Datasette 0.60 milestone Jan 13, 2022
@simonw
Copy link
Owner Author

simonw commented Jan 14, 2022

That's because that page has this unnecessary hidden form field:

<input type="hidden" name="_city_id__gt" value="1">

That field is added by this bit in the template:

{% for key, value in form_hidden_args %}
<input type="hidden" name="{{ key }}" value="{{ value }}">
{% endfor %}
<input type="submit" value="Apply">

Which is populated here:

form_hidden_args = []
for key in request.args:
if (
key.startswith("_")
and key not in ("_sort", "_search", "_next")
and not key.endswith("__exact")
):
for value in request.args.getlist(key):
form_hidden_args.append((key, value))

@simonw
Copy link
Owner Author

simonw commented Jan 14, 2022

Oh! This is because _city_id has a leading underscore (for testing purposes). I think I need to filter out any keys that contain __ in that case.

What happens to columns that contain a __? They shouldn't be reflected in the hidden arguments either - this code is really only supposed to catch things like _where and _m2m_through= and _col - basically most of the list on https://docs.datasette.io/en/stable/json_api.html#special-table-arguments

simonw added a commit that referenced this issue Jan 14, 2022
@simonw
Copy link
Owner Author

simonw commented Jan 14, 2022

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant