Autocomplete does not work with Django 3.2 #963

FrozenAlex · 2021-05-02T17:25:50Z

VERSIONS:
Grappelli 2.15.1, Django 3.2
STATICFILES: WARNINGS,
Details:
I see a lot of errors with text like this

Found another file with the destination path 'admin/js/autocomplete.js'. It will be ignored since only the first encountered file is collected. If this is not what you want, make sure every static file has a unique path.

JAVASCRIPTS: ERROR
Failed to load resource: the server responded with a status of 403 (Forbidden)
CUSTOMIZATIONS: NONE

I upgraded django from 2.2 to 3.2 and it broke the autocomplete on related fields it just always says 403 and gives the following error:

Forbidden (Permission denied): /admin/autocomplete/
Traceback (most recent call last):
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/utils/datastructures.py", line 76, in __getitem__
    list_ = super().__getitem__(key)
KeyError: 'app_label'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/contrib/admin/views/autocomplete.py", line 61, in process_request
    app_label = request.GET['app_label']
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/utils/datastructures.py", line 78, in __getitem__
    raise MultiValueDictKeyError(key)
django.utils.datastructures.MultiValueDictKeyError: 'app_label'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/contrib/admin/sites.py", line 250, in wrapper
    return self.admin_view(view, cacheable)(*args, **kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/contrib/admin/sites.py", line 232, in inner
    return view(request, *args, **kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/contrib/admin/sites.py", line 417, in autocomplete_view
    return AutocompleteJsonView.as_view(admin_site=self)(request)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/views/generic/base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/views/generic/base.py", line 98, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/contrib/admin/views/autocomplete.py", line 20, in get
    self.term, self.model_admin, self.source_field, to_field_name = self.process_request(request)
  File "/home/alex/Project/env/lib/python3.8/site-packages/django/contrib/admin/views/autocomplete.py", line 65, in process_request
    raise PermissionDenied from e
django.core.exceptions.PermissionDenied
[02/May/2021 20:11:33] "GET /admin/autocomplete/ HTTP/1.1" 403 135

This ticket seems to be related to the issue
https://code.djangoproject.com/ticket/32659?cversion=2&cnum_hist=2

Downgrade to 3.1 fixes the issue.

The text was updated successfully, but these errors were encountered:

sehmaschine · 2021-05-03T13:57:53Z

not sure I can help here. the warning looks "ok" (since we overwrite django files), you can ignore that. if the JS is not being loaded properly, there will be errors of course. I'm not sure why you get a 403, but it's definitely not related to grappelli.

int-ua · 2021-05-05T09:06:50Z

Django 3.2, Grappelli 2.15.1

~~It looks like a problem is the fact that there are two jQuery versions being loaded, 3.3.1 from Grappelli and 3.5.1 from Django 3.2.~~

https://code.djangoproject.com/ticket/32659

It's definitely a Grappelli problem at least partially unless Grappelli does not intend to support ModelAdmin.autocomplete_fields. And while on usual fields it's possible to replace with raw_id_fields, the https://github.com/farhan0581/django-admin-autocomplete-filter may never work without it. ~~The simple solution may be updating jQuery in Grappelli, not sure if that will work.~~

Again, it does work if I remove grappelli from installed apps and do collectstatic and it stops working immediately after I add it back and perform collectstatic again.

int-ua · 2021-05-05T09:09:50Z

You get 403 because with Grappelli there are no app_label, model_name and field_name in GET parameters, only term.

int-ua · 2021-05-05T14:03:47Z

And they are not there because /static/admin/js/autocomplete.js is outdated. Please sync it with upstream.

Workaround for anyone else experiencing this: create an empty Django app with the upstream version of autocomplete.js in static/admin/js and put it before grappelli in the INSTALLED_APPS. Edit: I've also included override for emptying jquery-migrate and update for grappelli/jquery to 3.5.1. Let's see what will break.

Please comment if you discover more outdated JS files that have to be synced with upstream. I guess it's likely there are more of them.

sehmaschine · 2021-05-05T14:27:11Z

just to clarify. which "autocompletes" do you refer to? the django autocompletes or the grappelli autocompletes?

int-ua · 2021-05-05T14:28:52Z

Django autocompletes, the ones used in https://github.com/farhan0581/django-admin-autocomplete-filter Sorry, that was ambiguous, yes.

sehmaschine · 2021-05-05T14:31:33Z

thanks, I see. I'll check that.

int-ua · 2021-05-05T14:50:12Z

Thank you! ~~Just noticed that admin/filter_listing.html doesn't support these Django autocompletes, shows only All option without the select element. Should I create a separate issue for that?~~

sehmaschine · 2021-05-05T14:53:39Z

not sure I can follow. filter with autocompletes? does django support that? if so, I haven´t been aware. but yes, please open a separate ticket ...

int-ua · 2021-05-05T15:10:01Z

Not by default, only with the aforementioned django-admin-autocomplete-filter. So it's more like support for that module rather than a bug. I'll try to do it with a custom change_list_filter_template first.

Kiougar · 2021-05-19T08:28:12Z

Any updates on this?

sehmaschine · 2021-05-19T08:39:58Z

We'll look into this as soon as we have some spare time. If you're in a hurry, a PR might help.

Kiougar · 2021-05-19T09:33:15Z

Any direction for creating a PR? Anything I should know?

wangzhaohe · 2021-05-28T09:18:51Z

Just use Django3.1

Django 3.2 still does not work.

ahmedsafadii · 2021-06-04T13:38:51Z

no fix 3.2.4

ajakubo1 · 2021-07-09T13:33:16Z

Just to clarify as I encountered exactly the same problem (maybe that will help pinpointing a solution):

$ pip freeze | grep Django
Django==3.2.4
$ pip freeze | grep -i grappelli
django-grappelli==2.15.1
$ find ./lib/ | grep autocomplete.js
./lib/python3.8/site-packages/django/contrib/admin/static/admin/js/autocomplete.js
./lib/python3.8/site-packages/grappelli/static/admin/js/autocomplete.js

This is autocomplete.js code from grappelli:

(function($) {
    'use strict';
    var init = function($element, options) {
        var settings = $.extend({
            ajax: {
                data: function(params) {
                    return {
                        term: params.term,
                        page: params.page
                    };
                }
            }
        }, options);
        $element.select2(settings);
    };

    $.fn.djangoAdminSelect2 = function(options) {
        var settings = $.extend({}, options);
        $.each(this, function(i, element) {
            var $element = $(element);
            init($element, settings);
        });
        return this;
    };

    $(function() {
        // Initialize all autocomplete widgets except the one in the template
        // form used when a new formset is added.
        $('.admin-autocomplete').not('[name*=__prefix__]').djangoAdminSelect2();
    });

    $(document).on('formset:added', (function() {
        return function(event, $newFormset) {
            return $newFormset.find('.admin-autocomplete').djangoAdminSelect2();
        };
    })(this));
}(django.jQuery));

this is the code from django3:

'use strict';
{
    const $ = django.jQuery;
    const init = function($element, options) {
        const settings = $.extend({
            ajax: {
                data: function(params) {
                    return {
                        term: params.term,
                        page: params.page,
                        app_label: $element.data('app-label'),
                        model_name: $element.data('model-name'),
                        field_name: $element.data('field-name')
                    };
                }
            }
        }, options);
        $element.select2(settings);
    };

    $.fn.djangoAdminSelect2 = function(options) {
        const settings = $.extend({}, options);
        $.each(this, function(i, element) {
            const $element = $(element);
            init($element, settings);
        });
        return this;
    };

    $(function() {
        // Initialize all autocomplete widgets except the one in the template
        // form used when a new formset is added.
        $('.admin-autocomplete').not('[name*=__prefix__]').djangoAdminSelect2();
    });

    $(document).on('formset:added', (function() {
        return function(event, $newFormset) {
            return $newFormset.find('.admin-autocomplete').djangoAdminSelect2();
        };
    })(this));
}

I guess an easy fix would be to add (1):

                        app_label: $element.data('app-label'),
                        model_name: $element.data('model-name'),
                        field_name: $element.data('field-name')

to Grappelli lib, or (2) copy the code from django3. Or (3)... I think removing autocomplete.js from Grappelli will also work (tested it locally and the original Django file was loaded instead of Grappelli version when I did python manage.py collectstatic -c). I guess I can open a PR with one of the "fixes", just the question is - which one (1, 2 or 3)?

maximdeclercq · 2021-07-10T13:48:34Z

I can confirm that removing the file fixes it, however, the visuals are still bugged.

sehmaschine · 2021-07-10T13:51:42Z

@maximdeclercq what you're showing (visually) looks like a Grappelli autocomplete, not like a Django autocomplete field. Not sure if that's the result of removing the Django file, but it certainly does not look right. There are 2 different autocompletes: the Django autocomplete (which relies on the mentioned file) and the Grappelli autocomplete – and we do not style the first one.

maximdeclercq · 2021-07-10T13:55:47Z

This is the Django autocomplete. It is defined like this.

class MyAdmin(admin.ModelAdmin):
    ...
    autocomplete_fields = ("user",)

Removing the file shouldn't change anything I think, because the only real difference were the fields that were missing from the ajax request.

sehmaschine · 2021-07-10T13:59:30Z

I see. Then it´s interesting that it doesn't look like the Django autocomplete. There still has to be another issue with this.

ajakubo1 · 2021-07-10T14:12:21Z

Maybe there's also a .css file for autocomplete that's being overridden?

maximdeclercq · 2021-07-10T14:25:35Z

The problem seems to be that these CSS variables are not defined. I'm looking into it.

maximdeclercq · 2021-07-10T16:47:41Z

Adding admin/css/base.css did not seem like the right choice since it conflicted a lot. I tweaked a bit with the grappelli CSS and managed to get this layout.

This looks fine to me. If you are happy with it you can merge my PR.

sehmaschine · 2021-07-10T16:58:01Z

Thanks! I’ll take a look on Monday.

…

On Sat 10. Jul 2021 at 18:47, Maxim De Clercq ***@***.***> wrote: Adding admin/css/base.css did not seem like the right choice since it conflicted a lot. I tweaked a bit with the grappelli CSS and managed to get this layout. [image: preview] <https://camo.githubusercontent.com/20198b1b4da22877cac644959c8625d271cf6edd904c5986e8de8ee39003a299/68747470733a2f2f692e696d6775722e636f6d2f63597455695a4b2e706e67> This looks fine to me. If you are happy with it you can merge my PR. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#963 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABNIWH2SEJPJXZH7URMWX3TXB2TRANCNFSM437S7E4Q> .

sehmaschine · 2021-07-12T14:42:12Z

@maximdeclercq Looks ok with ForeignKey, but not with M2M (that said, M2M looks kind of broken with the vanilla admin as well – but maybe that's just my view). I would say the PR is fine since it seems to work now. Styling these fields is almost impossible IMO.

selected-pixel-jameson · 2021-08-13T14:46:30Z

Django 3.2 is still not working. I had to revert to 3.1. I'm not using grapelli at all. Getting the exacts errors seen here.

sehmaschine · 2021-08-14T10:21:02Z

@selected-pixel-jameson what do you mean with "I'm not using grappelli at all"? because if that's the case, you're most probably wrong with this issue tracker. besides, the problem should be solved with current master (not the pypi version though).

selected-pixel-jameson · 2021-08-18T12:16:26Z

@sehmaschine I don't have Grappelli installed. I'm not using it. How do I grab the master using pip instead of the published version?

sehmaschine · 2021-08-18T12:18:37Z

@selected-pixel-jameson If you're not using grappelli ... why do you want to install it? And why do you comment on an issue which you obviously don't have (since you're not using grappelli)? sorry, but I don't get it.

selected-pixel-jameson · 2021-08-18T12:20:53Z

@sehmaschine I'm sorry. For some reason I thought this was the DAL repo. I apologize for the confusion.

waketzheng · 2021-10-19T09:25:56Z

This worked for me:

python manage.py collectstatic

Then activate the Disable cache in webbrowser develpor tool and refresh the page.

sdsy888 · 2021-12-09T07:24:34Z

Disable cache

tried this, but no luck for me.

waiting for the devs to check where the real problem is.

sehmaschine · 2021-12-27T13:21:06Z

Can someone please explain what you mean with "not working"? I tried this several times now and I can´t see an issue (besides the look&feel which we currently not plan to fix with the Django autocomplete).

Here are 3 Screenshots with my setup ...

The first one shows the Django version (without Grappelli):

Now with Grappelli installed, but still using the Django autocompletes:

And now the Grappelli version with the Grappelli autocompletes:

sehmaschine · 2021-12-27T14:26:40Z

Autocomplete does not work with Django 3.2 #963

Autocomplete does not work with Django 3.2 #963

Comments

FrozenAlex commented May 2, 2021 • edited Loading

sehmaschine commented May 3, 2021

int-ua commented May 5, 2021 • edited Loading

int-ua commented May 5, 2021

int-ua commented May 5, 2021 • edited Loading

sehmaschine commented May 5, 2021

int-ua commented May 5, 2021

sehmaschine commented May 5, 2021

int-ua commented May 5, 2021 • edited Loading

sehmaschine commented May 5, 2021

int-ua commented May 5, 2021

Kiougar commented May 19, 2021

sehmaschine commented May 19, 2021

Kiougar commented May 19, 2021

wangzhaohe commented May 28, 2021

ahmedsafadii commented Jun 4, 2021

ajakubo1 commented Jul 9, 2021

maximdeclercq commented Jul 10, 2021

sehmaschine commented Jul 10, 2021

maximdeclercq commented Jul 10, 2021

sehmaschine commented Jul 10, 2021

ajakubo1 commented Jul 10, 2021

maximdeclercq commented Jul 10, 2021

maximdeclercq commented Jul 10, 2021

sehmaschine commented Jul 10, 2021 via email

sehmaschine commented Jul 12, 2021

selected-pixel-jameson commented Aug 13, 2021

sehmaschine commented Aug 14, 2021

selected-pixel-jameson commented Aug 18, 2021

sehmaschine commented Aug 18, 2021

selected-pixel-jameson commented Aug 18, 2021

waketzheng commented Oct 19, 2021

sdsy888 commented Dec 9, 2021

sehmaschine commented Dec 27, 2021 • edited Loading

sehmaschine commented Dec 27, 2021

sehmaschine commented Dec 28, 2021

sehmaschine commented Jan 6, 2022

FrozenAlex commented May 2, 2021 •

edited

Loading

int-ua commented May 5, 2021 •

edited

Loading

int-ua commented May 5, 2021 •

edited

Loading

int-ua commented May 5, 2021 •

edited

Loading

sehmaschine commented Dec 27, 2021 •

edited

Loading